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TRAINING  DIGITAL  SKILLS  IN  DISTRIBUTED  CLASSROOM  ENVIRONMENTS:  A 
BLENDED  LEARNING  APPROACH 

EXECUTIVE  SUMMARY 


Research  Requirement: 

The  present  research  was  conducted  as  part  of  the  U.S.  Army  Research  Institute’s  (ARI) 
internal  workprogram  that  is  focused  on  science  of  learning  objectives.  As  the  U.S.  Army 
Training  and  Doctrine  Command  (TRADOC)  faces  resource  challenges  in  developing  and 
sustaining  both  effective  and  efficient  institutional  training,  it  has  sponsored  ARI  to  investigate 
different  blended  learning  approaches  for  training  within  the  Schoolhouse.  As  there  is  a  need  to 
determine  effective  methods  for  using  distributed  learning  (dL)  to  train  different  echelons  and 
skills,  we  investigated  the  blended  learning  approaches  used  to  train  digital  skills  in  distributed 
classroom  environments  and  the  students’  reactions  to  this  type  of  instruction.  Specific  goals  of 
the  present  research  were  to  1)  report  lessons  learned  from  instructors  in  a  distributed  blended 
learning  environment;  2)  report  student  reactions  to  this  type  of  learning  environment;  and  3) 
compare  the  training  approaches  with  those  reported  for  traditional  classroom  environments. 

Procedure: 

To  accomplish  the  goals  of  the  research,  we  collected  observational  and  interview/survey 
data  from  two  instructors  of  a  dL  Force  XXI  Battle  Command  Brigade  and  Below  (FBCB2) 
course  at  the  Battle  Command  Training  Center  (BCTC),  Camp  Dodge,  IA  and  from  the  students 
of  these  classes  at  remote  locations.  Observers  used  a  structured  protocol  to  collect  data  on  the 
training  environment,  instructional  activities,  and  training  techniques  that  were  used.  A  portion 
of  these  data  were  used  to  compare  blended  learning  instructional  approaches  with  traditional 
classroom  training. 

Findings: 

Both  the  dL  instructors  and  students  acknowledged  substantial  value  in  the  blended 
approach  (video  teletraining  with  computer  software)  for  training  digital  skills.  The  students 
offered  some  good  suggestions  for  improving  the  training  in  distributed  environments  such  as 
having  hands-on  training  with  tactical  equipment  and  having  additional  training  aides  (acronym 
list,  commander’s  guide,  train-the-trainer  materials,  etc.).  Additionally,  major  advantages  and 
disadvantages  of  the  dL  method  of  instruction  are  noted  in  the  report. 

Overall,  the  findings  indicate  that  the  dL  instructors  were  able  to  adapt  to  using  the 
technology  to  teach  FBCB2  operations,  resulting  in  training  that  is  very  comparable  to  the 
traditional  courses.  The  positive  student  comments  and  similar  training  approaches  and  topical 
coverage  support  these  findings. 

The  dL  instructors  applied  a  variety  of  techniques  to  convey  course  content,  maintain  the 
students’  motivation,  and  respond  to  students’  problems  and  concerns.  Several  techniques  and 
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training  aids  were  identified  as  particularly  effective  for  distributed  classroom  environments, 
including  selecting  one  of  the  more  capable  students  to  serve  as  a  demonstrator,  encouraging 
peer  coaching,  taking  control  of  a  student’s  mouse  to  ensure  proper  execution  of  procedural 
steps,  and  conducting  “mini-exercises”  to  reinforce  key  skills  or  tasks  and  enable  the  instructors 
to  perform  structured  learning  checks. 

Although  the  overall  patterns  of  dL  activities  and  techniques  were  similar  to  those  found 
in  traditional  classrooms,  specific  adaptations  occurred  to  accommodate  the  dL  environment. 
Overall,  the  dL  instructors  spent  more  time  covering  selected  topics  than  the  instructors  in  the 
traditional  classes.  The  absence  of  face-to-face  interaction  may  have  led  the  instructors  to 
conduct  more  learning  checks  to  generate  feedback  on  student  progress,  which  may  account  for 
some  of  the  additional  time.  The  checks  on  learning,  review  of  material,  and  testing  are  all 
effective  ways  to  ensure  that  students  understand  the  material  and  are  learning  the  skills  and 
should  be  sustained.  Although  some  differences  were  noted  between  the  dL  and  traditional 
classes,  these  may  have  been  due  to  other  factors  besides  the  training  medium  (e.g.,  background 
of  students  in  the  class,  etc.). 

The  findings  suggest  that  the  right  mix  of  training  techniques  depends  on  the  skill  level  of 
the  students,  which  varies  from  class  to  class.  A  good  approach  is  to  train  instructors  to  be 
knowledgeable  about  the  different  training  techniques  and  when  to  use  them.  With  this  in  mind, 
several  specific  recommendations  for  dL  instructors  were  made  in  the  areas  of  (a)  leveraging 
student  strengths,  (b)  emphasizing  problem-centered  instruction,  and  (c)  leveraging  training  aids. 

Utilization  and  Dissemination  of  Findings: 

While  the  sampling  approach  was  limited,  the  results  provide  insights  and  lessons  for 
ensuring  effective  courses  for  FBCB2  operators.  The  findings  contribute  to  the  Army’s  efforts  to 
enhance  blended  learning  solutions  to  meet  critical  training  needs.  Many  of  the 
recommendations  are  applicable  to  both  distributed  blended  learning  environments  and 
traditional  classroom  settings,  and  we  encourage  instructors  to  use  these  ideas  to  make  small 
changes  to  their  training  that  may  result  in  big  differences  in  their  students’  proficiency  levels. 
Major  findings  were  briefed  to  the  Project  Officer  and  digital  instructors  of  the  BCTC  at  Camp 
Dodge,  IA. 
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TRAINING  DIGITAL  SKILLS  IN  DISTRIBUTED  CLASSROOM  ENVIRONMENTS:  A 

BLENDED  LEARNING  APPROACH 


INTRODUCTION  AND  BACKGROUND 

The  present  research  was  conducted  as  part  of  the  U.S.  Army  Research  Institute’s  (ARI) 
internal  workprogram  that  is  focused  on  science  of  learning  objectives.  As  the  U.S.  Army 
Training  and  Doctrine  Command  (TRADOC)  faces  resource  challenges  in  developing  and 
sustaining  both  effective  and  efficient  institutional  training,  it  has  sponsored  ARI  to  investigate 
different  blended  learning  approaches  for  training  within  the  Schoolhouse.  To  identify  learning 
science  findings  and  technologies  for  training  Soldiers,  ARI  conducted  a  Science  of  Learning 
Workshop  in  2006  that  brought  together  experts  from  industry,  academia,  and  other  military 
services  (see  Quinkert,  Morrison,  Fletcher,  Moses,  &  Roberts,  2007).  From  this  workshop,  the 
ARI-Fort  Benning  unit  developed  a  workprogram  to  conduct  blended  learning  research  and 
leverage  lessons  learned  from  science  and  technology  work  to  the  Schoolhouse.  The  present 
research  addresses  several  of  the  objectives  of  this  workprogram: 

•  Examine  effective  methods  for  using  distributed  learning  (dL)  in  Army  institutional 

training  and  successful  ways  to  incorporate  other  emerging  training  technologies. 

•  Investigate  exemplar  blended  learning  solutions  for  technical  training  and  leader  training. 

•  Identify  and  leverage  best  dL  practices  from  Army  schools,  other  services,  and  industry. 

Current  Institutional  Training  Environment 

At  the  Science  of  Learning  Workshop,  General  William  S.  Wallace,  Commanding 
General,  TRADOC,  discussed  a  transformation  that  is  occurring  between  the  Institutional  and 
Operating  forces  (see  Quinkert  et  al.,  2007;  Wallace,  2006). 1  Traditionally,  there  has  been  a 
linear  flow  of  forces  and  doctrine  from  the  Institutional  to  the  Operational  Army.  However,  this 
model  does  not  allow  for  feedback  in  the  opposite  direction  such  as  improving  training  and 
doctrine  by  applying  lessons  learned.  Thus,  a  dynamic  model  has  evolved  such  that  TRADOC, 
as  the  Generating  Force,  is  fully  integrated  with  the  Operational  Force  and  is  responsible  for 
training  leaders  and  Soldiers  to  perform  in  the  current  environment  as  well  as  identifying  the 
training  gaps  to  respond  effectively  to  future  threats. 

Two  major  factors  have  put  pressure  on  TRADOC  to  produce  more  trained  leaders  and 
Soldiers  -  the  Nation  at  War  and  the  Army  Force  Generation  Model  (ARFORGEN).“  In  addition 
to  responding  to  increasing  output  requirements,  TRADOC  is  facing  significant  budgetary  and 
personnel  reductions.  To  meet  the  training  needs  within  these  resource  constraints,  TRADOC 
has  changed  the  way  they  implement  mobile  training  teams  (MTTs)  and  blended  distance 
learning  and  traditional  classroom  instruction.  Although  TRADOC  has  supported  the  use  of 
blended  learning  approaches  in  Army  institutional  training,  there  is  a  need  to  determine  effective 


The  information  contained  in  this  paragraph  and  the  following  paragraph  is  summarized  from  the  Quinkert  et  al. 
(2006)  and  Wallace  (2006)  reports. 

2  “ARFORGEN  is  the  structured  progression  of  increased  unit  readiness  over  time  that  results  in  recurring  periods  of 
availability  of  trained,  ready,  and  cohesive  units  prepared  for  operational  deployment  in  support  of  civil  authorities 
and  combatant  commanders”  (Department  of  the  Army,  2005,  as  cited  in  Wallace,  2006). 
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methods  for  using  dL  to  train  different  echelons  and  skills.  Thus,  the  purpose  of  this  report  is  to 
provide  information  on  the  blended  learning  approaches  used  to  train  digital  skills  in  distributed 
classroom  environments  and  on  the  students’  reactions  to  this  type  of  instruction. 

Blended  Learning  Approaches 

Blended  learning  instructional  approaches  are  defined  as  those  which  combine  different 
training  media  (technologies,  activities,  types  of  events)  to  create  an  optimum  training  program 
for  a  specific  audience  (Bersin,  2004).  Typically,  the  term  blended  learning  is  used  to  describe 
instruction  that  combines  face-to-face  (F2F)  instruction  with  computer-mediated  instruction 
(Bonk  &  Graham,  2006).  The  widespread  adoption  and  availability  of  digital  learning 
technologies  have  made  synchronous,  or  high-fidelity,  interactions  in  a  distributed  environment 
possible  (Bonk  &  Graham,  2006).  Thus,  human  interactions  that  once  only  occurred  in  F2F 
environments  are  now  facilitated  in  real  time  via  computer- supported  collaboration,  virtual 
communities,  instant  messaging,  live- virtual-constructive  exercises,  etc. 

Instructors,  trainers,  and  learners  may  choose  blended  learning  approaches  for  at  least 
three  overlapping  reasons:  improved  pedagogy,  increased  access  and  flexibility,  and  increased 
cost  effectiveness  (Bersin,  2004;  Bonk  &  Graham,  2006).  Specifically,  blended  learning 
approaches  may  increase  the  level  of  active  learning  strategies  or  experiential  learning,  leading  to 
greater  mastery  of  skills  (proficiency  +  retention),  productivity,  and  retention.  Some  research 
supports  this  idea  that  there  is  increased  learning  and  retention  compared  to  traditional  lecture  or 
completely  distributed  formats  (Bersin,  2004;  Bonk  &  Graham,  2006).  Additional  empirical 
investigations  are  needed  to  determine  the  generalizability  of  these  positive  effects  across  job 
tasks  and  organizational  contexts  as  well  as  provide  results  from  cost/benefit  comparisons  of 
different  instructional  media.  However,  if  training  developers  align  their  program  with  strategic 
objectives  and  posit  blended  learning  as  a  solution  to  close  long-term  skill  gaps,  then  they  may 
be  justified  in  building  a  more  expensive  program.  There  is  some  evidence  to  suggest  that 
developers  who  have  taken  this  approach  have  achieved  positive  individual  and  organizational 
outcomes  such  as  effective  transformation  of  processes  and  rollouts  of  new  products,  increased 
productivity  and  job  satisfaction  and  reduced  turnover  (Bersin,  2004;  Bonk  &  Graham,  2006).  It 
is  important  to  note  that  while  one  goal  of  blended  learning  training  programs  is  to  reach  a  wider 
audience  at  a  lower  cost,  “the  business  impact  of  reducing  the  cost  of  training  is  virtually  zero” 
(Bersin,  2004,  p.  20).  In  other  words,  rather  than  only  examining  the  cost  effects  of  blended 
training  programs,  these  programs  should  be  evaluated  according  to  whether  they  solved  high- 
impact  problems.  Bersin  (2004)  suggests  that  a  1%  increase  in  productivity  has  more  than  10 
times  the  financial  impact  as  a  1%  decrease  in  training  costs. 

The  typical  approach  to  developing  a  blended  learning  course  reflects  a  program  flow 
model  in  which  some  of  the  F2F  interactions  in  an  instructor-led  training  program  are  replaced 
with  self-study  or  e-leaming  activities  (Bersin,  2004).  Thus,  different  media  are  integrated  into  a 
step-by-step  series  of  events  which  are  usually  delivered  in  cycles  of  self-study,  live  events, 
check-in  processes  with  the  instructor,  and  assessments.  Bersin  (2004)  reported  that  many 
different  organizations  have  adopted  this  approach  and  suggested  that  training  programs  should 
be  designed  such  that  one  or  two  hours  of  self-study  are  followed  by  a  live  check-in  process  so 
that  instructors  can  better  track  progress.  This  model  fits  into  the  normal  flow  of  classroom 
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training  that  most  employees  expect  and  reflects  most  instructional-design  paradigms  of 
leam/try/assess  as  well  as  the  Army  Learning  Model  (ALM;  e.g.,  TRADOC  Analysis  Center, 
2007).  For  example,  the  ALM  involves  three  phases:  1)  Individual  Learning  Preparation 
consisting  of  dL  which  could  be  required  before,  during,  or  after  F2F  instruction,  2)  Collective 
Learning  Synergy  consisting  of  guided  experiential  learning  approaches  [blended  training  (F2F, 
dL,  simulations)  reflecting  a  process  of  instructor-led  demonstrations,  practice  sessions,  and 
feedback],  and  3)  Learning  Reinforcement  consisting  of  reinforcement  training  at  the  unit 
supported  by  dL  and  job  aids. 

In  the  military,  blended  learning  approaches  have  been  mostly  used  in  specialized  skill 
training  that  “provides  personnel  with  initial  job  qualification  skills  and  new  or  higher  levels  of 
skill  in  military  specialties  to  meet  specific  job  requirements”  (Wisher,  2006,  p.  522).  An 
example  of  an  Army  course  that  uses  a  blend  of  instructional  techniques  is  the  Maneuver 
Captains  Career  Course  for  Army  National  Guard  or  Army  Reserve  Soldiers  (MC3-RC). 
Although  the  design  of  the  course  has  changed  somewhat  since  Wisher’s  (2006)  description,  it 
still  consists  of  asynchronous,  synchronous,  and  face-to-face  instruction.  The  asynchronous 
instruction  provides  the  Captains  with  doctrinal  knowledge  delivered  over  the  Internet  while  the 
synchronous  instruction  allows  the  Captains  to  interact  with  each  other  from  remote  locations  in 
a  virtual  tactical  operations  center.  The  Captains  use  collaboration  tools  such  as  chat  rooms,  a 
whiteboard,  a  book  shelf,  shared-text  application,  and  three-dimensional  terrain  tools  to  solve 
problems  collectively  and  develop  mission  planning  products. 

Of  importance  to  the  present  research  is  an  Army  National  Guard  course  that  has  blended 
instructional  approaches  to  train  operator  skills  for  a  digital  command  and  control  system,  Force 
XXI  Battle  Command  Brigade  and  Below  (FBCB2),  which  is  a  component  of  the  Army  Battle 
Command  System  (ABCS).  The  course  is  designed  for  distributed  instruction  such  that  the 
Soldiers  are  in  computerized  classrooms  in  their  home  states,  remote  from  the  instructor,  who  is 
located  at  the  Battle  Command  Training  Center  (BCTC),  Camp  Dodge,  IA. 

Factors  Affecting  Blended  Learning  Outcomes 

Individual  characteristics.  The  experience  level  of  an  individual  student  significantly 
influences  the  effectiveness  of  particular  blended  learning  approaches.  For  example, 
instructional  approaches  appropriate  for  novices  differ  from  those  appropriate  for  more 
experienced  learners  (Clark  &  Wittrock,  2000).  When  the  learners’  background  knowledge  is 
extensive,  the  use  of  guided-discovery  approaches  in  the  beginning  of  the  instruction  where 
learners  participate  in  a  problem-solving  exercise  and  then  receive  feedback  and  resources  to 
improve  the  learners’  outcomes  may  be  more  effective  than  directive  techniques  known  as 
learning  by  telling.  Experienced  learners  benefit  more  in  terms  of  comprehension 
(understanding)  and  generation  (creation  of  meaning)  when  they  solve  real-world  problems  by 
using  their  prior  knowledge  and  experience  to  make  sense  of  new  information  and  develop  new 
skills  (Clark  &  Wittrock,  2000).  Some  evidence  supports  this  idea-  initial  entry  Soldiers 
(Privates)  preferred  and  performed  best  in  an  instructor-centered  learning  context  (lessons 
followed  by  exercises  or  exploration  activities)  compared  to  Infantry  officers  who,  with  more 
education,  computer  skill,  and  Army  experience  and  knowledge,  preferred  to  select  their  own 
learning  strategy  (Dyer,  Singh,  &  Clark,  2005).  Dyer  et  al.  explained  that  the  initial  entry 
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Soldiers  lacked  the  requisite  metacognitive  skills  to  take  advantage  of  the  self-select  method. 

That  is,  the  Soldiers  did  not  know  what  types  of  training  worked  best  for  them,  thus,  they  were 
unable  to  identify  these  methods  during  the  training  process.  On  the  other  hand,  the  officers 
were  more  consistent  in  their  training  strategy  selections,  showing  more  self -regulatory  skills  by 
staying  with  the  training  modes  that  worked  best  for  them. 

Media  selection  &  sequencing  of  instruction.  In  addition  to  the  effects  of  individual 
characteristics,  instructional  media  indirectly  influence  learning  outcomes  by  affecting  the 
cognitive  processes  responsible  for  comprehension,  generation,  and  transfer  of  training.  That  is, 
media  support  the  cognitive  processes  reflecting  the  interpretation  of  the  performance  goal,  the 
retrieval  of  task-relevant  knowledge,  the  execution  of  new  procedural  knowledge,  performance 
monitoring,  and  the  diagnosis  and  correction  of  sources  of  error  in  performance  (Sugrue  &  Clark, 
2000).  Thus,  media  should  be  selected  to  compensate  for  weaknesses  in  trainees’  abilities  to 
perform  such  processes  by  elaborating  on  task  goals,  providing  task-related  information  and 
opportunities  for  practice,  monitoring  performance,  and  diagnosing  and  correcting  errors  (Sugrue 
&  Clark,  2000). 

There  is  a  debate  among  academics  regarding  the  points  in  the  curriculum  at  which 
different  media  should  be  used  to  provide  students  with  relevant  information  about  the  course 
topics.  As  such,  the  sequencing  of  learning  events,  including  the  use  of  media  for  “telling” 
instruction  such  as  presentations  and  lectures,  demonstrations,  and  experiential  activities  as  well 
as  how  much  information  is  given  to  students,  also  is  a  source  of  contention  (cf.,  Hmelo-Silver, 
Duncan,  &  Chinn,  2007;  Kirshner,  Sweller,  &  Clark,  2006;  Mayer,  2004;  Schwartz  &  Bransford, 
1998;  Sugrue  &  Clark,  2000).  The  choice  of  such  sequencing  may  depend  in  part  on  both  the 
individual  characteristics  of  the  students  as  well  as  the  type  of  skills  developed  by  the  course. 

For  example,  as  described  above,  novices  may  leam  best  if  they  are  first  shown  a  complete 
worked  example  of  the  content  or  provided  with  a  lecture  while  more  experienced  students  may 
leam  best  if  they  can  shape  their  own  learning  experiences  (Dyer  et  al.,  2005).  However,  the 
amount  and  timing  of  information  required  by  both  novices  and  experts  will  likely  depend  on  the 
number  of  errors  they  make  during  task  performance  and  by  the  trainees’  own  interests  and 
perceived  needs  (Sugrue  &  Clark,  2000). 

Further,  the  sequencing  of  instruction  may  depend  in  part  on  the  types  of  skills  developed 
by  the  course.  For  instance,  to  develop  procedural  skills  such  as  assembling  and  disassembling  a 
weapon,  it  may  be  more  effective  to  first  provide  a  demonstration  of  the  correct  or  incorrect 
procedures  rather  than  asking  students  to  figure  out  how  to  do  this  on  their  own.  On  the  other 
hand,  to  develop  cognitive,  decision-making,  problem-solving  skills,  it  may  be  more  effective  for 
students  to  first  work  independently  to  try  to  solve  a  relevant  problem  prior  to  receiving  an 
instructor-led  demonstration  or  presentation  (cf.,  Hmelo-Silver  et  al.,  2007;  Schwartz  & 
Bransford,  1998).  Depending  on  how  different  military  systems  are  used  as  well  as  the  design 
and  complexity  of  the  system,  digital  skills  likely  fall  somewhere  between  purely  cognitive, 
decision-making  skills  and  procedural,  motor-centric  skills.  For  example,  when  commanders  use 
Army  systems  for  planning  purposes  they  may  use  more  cognitive  than  procedural  skills  because 
they  are  using  the  system  as  a  tool  to  wargame  different  courses  of  action  and  to  understand 
second-order  effects  of  their  decisions  (cf.,  James,  Dyer,  &  Wampler,  2008).  Thus,  they  will  use 
system  functions  in  different  ways  to  achieve  their  learning  goals.  On  the  other  hand,  operators 
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of  systems  on  different  platforms,  such  as  FBCB2,  may  use  more  procedural  than  cognitive  skills 
as  they  repeatedly  use  the  same  system  functions  to  perform  routine  tasks,  such  as  map,  message, 
and  administrative  functions  (cf.,  Bink,  Wampler,  Goodwin,  &  Dyer,  2008).  These  differences 
affect  the  type  (e.g.,  complexity,  level  of  student  engagement,  feedback),  authenticity  (i.e., 
realism),  amount  (more  practice  is  needed  to  transfer  to  novel  and  variable  job  contexts),  and 
sequencing  of  practice  sessions  embedded  in  training  contexts  (cf.,  Sugrue  &  Clark,  2000).  To 
understand  the  instructional  approaches  employed  when  training  digital  skills,  some  researchers 
have  suggested  conceptual  links  to  psychological  learning  theories  as  described  below. 

Digital  Skills  Training 

Digital  skills  are  those  that  are  needed  to  effectively  use  computer  software  involving 
data  entry  and  the  execution  of  commands  through  a  graphical  user  interface  (Goodwin,  2006). 
Digital  skills  are  discrete,  multi-step  procedures  as  operators  navigate  through  a  series  of  menus 
and  submenus  to  set  parameters  and  execute  commands  (Goodwin,  2006).  These  skills  are 
becoming  increasingly  important  as  the  Army  is  migrating  toward  total  force  fielding  of  the 
ABCS  (for  a  review  of  the  components  of  this  system  see  Leibrecht,  Goodwin,  Wampler,  & 
Dyer,  2007). 

In  a  review  of  training  principles  for  digital  training,  Goodwin  (2006)  provided  the 
following  three  conclusions  from  prior  research.  First,  completely  unguided  instruction  in  which 
no  training  materials  or  exercises  are  provided  to  the  participants  except  for  a  user  manual  is  the 
least  effective  means  for  training  digital  skills.  Second,  computerized  tutorials  are  not  as  good  as 
live  or  videotaped  instructor  demonstrations;  however,  Goodwin  noted  that  this  conclusion 
which  was  based  on  research  conducted  in  the  1980s  may  no  longer  be  valid  as  technology  has 
greatly  advanced  this  type  of  instruction.  Further,  some  variation  of  guided  exploration,  a 
technique  in  which  students  are  given  minimal  instruction  as  they  work  through  a  series  of 
exercises,  is  better  than  other  training  conditions  such  as  unguided  exploration,  behavioral 
modeling,  computerized  tutorial,  and  classroom  instruction  alone.  In  addition.  Dyer  and  Salter 
(2001)  conducted  an  experiment  to  assess  Soldiers’  performance  using  a  digital  map  interface 
and  found  that  Soldiers,  who  were  given  a  large  amount  of  information  before  they  could  apply 
it,  obtained  low  scores  on  the  assessment  exercises.  To  be  more  effective,  the  training  needed  to 
present  information  in  smaller  amounts  prior  to  the  application  of  the  material. 

Sanders  (2001)  and  Leibrecht  et  al.  (2007)  investigated  Army  digital  training  practices 
and  reported  the  techniques  currently  used  to  train  digital  skills  and  identified  the  learning 
theories  best  represented  by  those  techniques  (i.e.,  behaviorist,  cognitive,  and  constructivist). 
Specifically,  a  sample  of  the  training  from  four  different  ABCS  courses  was  observed.  For  the 
operator  courses,  training  techniques  reflecting  the  behaviorist  and  cognitive  theories  were 
observed  more  frequently  than  the  constructivist  theory.  That  is,  instructors  pointed  out  screen 
cues  and  prompts  (behaviorist)  and  conducted  guided  demonstration  (cognitive)  at  much  greater 
rates  than  encouraging  active  learning  (constructivist).  Leibrecht  et  al.  pointed  out  that  these 
findings  were  expected  because  the  courses  sampled  were  designed  for  novice  audiences. 
Although  this  general  pattern  was  consistent,  each  course  included  a  somewhat  unique  mix  of 
training  techniques,  and,  in  addition  to  the  more  traditional  training  approaches  (e.g.,  lecture, 
video,  and  practical  exercises),  some  instructors  employed  innovative  techniques  such  as  using 
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experienced  students  as  demonstrators,  distributing  information  to  students  through  digital 
media,  and  conducting  exercises  reflecting  real  world  missions.  Leibrecht  et  al.  recommended 
that  current  ABCS  courses  can  be  improved  by  incorporating  a  greater  use  of  constructivist 
techniques  such  as  conducting  exercises  which  require  students  to  apply  what  they  have  learned 
to  novel  problems  without  first  being  shown  a  step-by-step  demonstration.  Further,  by  relating 
new  material  to  general  knowledge  (cognitive  approach),  instructors  could  further  Soldiers’ 
understanding  of  the  new  procedures  by  relating  the  material  to  well-known  Windows  or 
Microsoft  applications. 

Present  Research 

As  there  is  a  need  to  determine  effective  methods  for  using  dL  to  train  different  echelons 
and  skills,  we  investigated  the  blended  learning  approaches  used  to  train  digital  skills  in 
distributed  classroom  environments  and  the  students’  reactions  to  this  type  of  instruction. 
Specific  goals  of  the  present  research  were  to  1)  report  lessons  learned  from  instructors  in  a 
distributed  blended  learning  environment;  2)  report  student  reactions  to  this  type  of  learning 
environment;  and  3)  compare  the  training  approaches  with  those  reported  by  Leibrecht  et  al. 
(2007)  for  traditional  classroom  environments. 

The  BCTC  at  Camp  Dodge,  IA  provides  a  unique  opportunity  in  which  to  address  the 
goals  of  the  present  research.  As  the  students  receive  the  digital  skills  training  remotely  via 
video-teletraining  (VTT)  and  with  sophisticated  computer  software  that  emulates  the  FBCB2 
system,  the  course  blends  VTT  with  dL  instruction.  As  such,  the  instructors  must  employ  a 
variety  of  techniques  to  cover  the  content,  maintain  the  students’  motivation,  and  sufficiently 
address  students’  problems  and  concerns.  To  accomplish  the  goals  of  the  research,  we  collected 
observational  and  interview/survey  data  from  instructors  and  students  of  two  blended/distributed 
FBCB2  classes. 

Specific  to  the  goal  of  comparing  blended  learning  instructional  approaches  with 
traditional  classroom  training,  we  first  studied  the  approaches  used  to  train  residential  ABCS 
courses  reported  by  Leibrecht  et  al.  (2007).  We  then  developed  observation  forms  based  on 
those  used  by  Leibrecht  et  al.  to  record  both  instructor  and  student  behaviors.  Since  we  expected 
that  a  significant  number  of  the  training  approaches  employed  by  the  instructors  would  reflect 
multiple  learning  theories,  we  used  the  Leibrecht  et  al.  forms  as  a  guide.  However,  we  did  not 
structure  our  results  according  to  the  specific  learning  theories  described  above. 
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METHOD 


Participants 

Instructors.  Given  that  all  of  the  instructors  at  the  BCTC  follow  the  same  POI,  we  felt 
that  interviewing  and  observing  two  instructors  would  be  sufficient.  We  felt  that  this  would 
enable  us  to  determine  if  there  were  any  major  differences  in  the  execution  of  the  POI  within  the 
practical  constraints  of  conducting  the  research  (i.e.,  time,  available  funds).  Table  1  summarizes 
the  experience  and  training  of  the  two  instructors.  Both  had  substantial  experience  as  FBCB2 
instructors,  especially  for  the  residential  course.  However,  neither  instructor  had  any  operational 
experience  with  FBCB2  nor  did  they  have  experience  teaching  other  ABCS  courses.  Perhaps 
most  notably,  neither  instructor  had  any  specific  training  in  distributed  instruction  techniques. 

Table  1 

Four  Dimensions  of  Instructor  Experience  ancl  Training 


Instructor 

Operational 

Experience 

FBCB2  Courses  Taught 

Other  ABCS 
Courses  Taught 

dL  Instructor 
Training 

Residential 

Distributed 

1 

None 

6 

3 

None 

None 

2 

None 

15 

2 

None 

None 

Students.  Sixteen  Soldiers  from  two  National  Guard  units  (Engineer  Battalion,  n  =  10, 
and  Infantry  Division,  n  =  6)  attended  the  FBCB2  course  at  two  different  locations.  The  majority 
of  Soldiers  had  recent  deployment  experience  (Engineer  unit:  average  months  =  12  for  Operation 
Iraqi  Freedom  and  12  for  Operation  Enduring  Freedom;  Infantry  Division:  average  months  =  14 
for  Operation  Iraqi  Freedom  only).  However,  only  three  Soldiers  had  experience  using  ABCS 
systems  either  in  deployment  operations  or  during  field  training  exercises:  one  Soldier  used 
FBCB2  plus  one  other  ABCS  system  in  deployment  operations  for  12  months;  a  second  Soldier 
used  another  ABCS  system  both  in  deployment  operations  for  96  hours  a  week  for  9  months  and 
during  field  training  exercises  for  two  weeks;  a  third  Soldier  used  a  subordinate  ABCS  system  in 
field  training  exercises  for  a  total  of  36  hours. 

Other  biographical  information  for  the  Soldiers  is  shown  in  Table  2.  Of  particular 
importance  to  the  present  research  is  that  the  majority  of  the  Soldiers  had  moderate  to  much 
computer  experience  and  felt  moderately  to  very  comfortable  using  computers. 

Finally,  the  majority  of  the  Soldiers  from  the  FBCB2-1  class  (Engineer  unit)  reported  that 
they  were  attending  the  course  to  prepare  for  deployment  while  the  Soldiers  from  the  FBCB2-2 
class  (Infantry  Division)  reported  that  they  were  attending  the  course  either  as  refresher  or  annual 
training  or  to  learn  the  system  so  that  they  could  train  their  Soldiers. 
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Table  2 

Student  Biographical  Information 


Biographical  Information 

Variable 

FBCB2-1 

FBCB2-2 

(n  =  10) 

(n  =  6) 

Rank/Grade 

1LT 

10% 

MAJ 

16.7% 

2LT 

10% 

CPT 

33.3% 

1SG 

10% 

SSG 

16.7% 

SFC 

20% 

SGT 

16.7% 

SSG 

30% 

PV2 

16.7% 

SGT 

10% 

SPC 

10% 

Average  Time  in 

15.0  years 

15.9  years 

Service 

Current  Duty 

Platoon  Leader 

10% 

Deputy  G6 

16.7% 

Position* 

Executive  Officer 

10% 

Operations  Officer 

16.7% 

First  Sergeant 

10% 

Provost  Marshall 

16.7% 

Platoon  Sergeant 

10% 

Communications  Sergeant 

16.7% 

Operations  NCO 

10% 

Imagery  Analyst 

16.7% 

Squad  Leader 

20% 

Aviation  Operations  Specialist 

16.7% 

Section  Sergeant 

10% 

Communications  Specialist 

10% 

NCOES 

1SG  Course 

10% 

1SG  Course 

0% 

ANCOC 

20% 

ANCOC 

0% 

BNCOC 

60% 

BNCOC 

33.3% 

PLDC/WLC 

60% 

PLDC/WLC 

33.3% 

Other 

10% 

Other 

0% 

OES 

OAC/CCC 

0% 

OAC/CCC 

50% 

OBC/BOLC  111 

20% 

OBC/BOLC  III 

33.3% 

OCS/ROTC 

10% 

OCS/ROTC 

50% 

Other 

10% 

Other 

16.7% 

Comfortable 

Not  At  All 

10% 

Not  At  All 

0% 

Using  a  Computer 

Slightly 

10% 

Slightly 

0% 

Moderately 

30% 

Moderately 

50% 

Very 

50% 

Very 

50% 

Computer 

None 

0% 

None 

0% 

Experience 

Little 

30% 

Little 

0% 

Moderate 

30% 

Moderate 

33.3% 

Much 

40% 

Much 

66.7% 

Note.  ANCOC  =  Advanced  Noncommissioned  Officer  Course;  BNCOC  =  Basic  Noncommissioned 
Officer  Course;  PLDC  =  Primary  Leadership  Development  Course;  WLC  =  Warrior  Leader  Course;  OAC 
=  Officer  Advanced  Course;  CCC  =  Captains  Career  Course;  OBC  =  Officer  Basic  Course;  BOLC  = 
Basic  Officer  Leader  Course;  OCS  =  Officer  Candidate  School;  ROTC  =  Reserve  Officer  Training  Corps. 
*  Does  not  equal  100%  due  to  missing  data. 


8 


Materials 


Instructor  observation  packet.  We  used  the  instructor  observation  forms  developed  by 
Leibrecht  et  al.  (2007)  to  conduct  the  instructor  observations  (Appendix  A).3  The  observation 
packet  consisted  of  three  forms:  1)  Part  I:  basic  classroom  details  (e.g.,  number  of  students, 
general  description  of  the  training  site);  2)  Part  IIA:  a  chronology  of  instructor  activities  and 
training  techniques,  Part  IIB:  a  sheet  to  record  practical  exercise  information;  and  3)  Part  III: 
prerequisite  training  and  course  objective  questions  and  end-of-course  assessment  questions. 

As  shown  in  Table  3,  eight  activities  were  selected  to  record  on  the  observation  form  in 
Part  IIA  of  the  packet  based  on  prior  observation  of  contemporary  ABCS  training;  the  form  did 
not  include  the  full  list  of  training  methods  described  in  TRADOC  Regulation  350-70  (U.S. 
Army  Training  and  Doctrine  Command,  1999).  Further,  as  shown  in  Table  3,  this  form  also 
included  10  specific  instructional  techniques  to  observe  based  on  prior  research  (Dyer  et  al., 
2005;  Sanders,  2001)  and  on  the  belief  that  these  behaviors  could  be  recorded  in  real-time  with  a 
high  degree  of  reliability;  the  observers  also  could  record  additional  behaviors  not  included  on 
the  list.  For  the  present  research  project,  specific  dL  techniques  were  added  to  the  observation 
form. 

Table  3 

Classroom  Activities  and  Training  Techniques  Selected  for  Observation  Purposes 


Activity 

Description 

Lecture 

Oral  presentation  of  information,  typically  accompanied  by  slides 

Video 

Film-based  presentation  of  real-world  scenes  and/or  animation 

Demonstration 

Illustration  of  steps/actions  by  demonstrator  (students  observe  only) 

Guided  Demonstration 

Performance  of  steps/actions  by  demonstrator  (students  replicate) 

Practical  Exercise 

Scenario-based  event  requiring  application  of  skills  and  knowledge 

Review 

Retrospective  summary  or  recapitulation  of  key  learning  points 

Test 

Formal  measurement  of  learning  by  means  of  quizzes,  exams,  etc. 

Break 

Temporary  suspension  of  formal  learning  activities 

Technique 

Description 

Emphasize  practice 

Provide  repeated  opportunities  to  perform  tasks  and  correct  errors 

Check  learning  progress 

Assess  learning  via  questions,  feedback,  and  performance  monitoring 

Point  to  screen  prompts 

Point  out  elements  in  slides  or  ABCS  screen  displays  to  guide  learning 

Use  memory  aids 

Provide  memory  prompts  and  mnemonics  to  facilitate  recall 

Provide  purpose  and  path 

Specify  course  benchmarks  or  topics,  and  maintain  path  awareness 

Relate  to  military  operations 

Put  system  functions  in  context  of  military  knowledge  or  operations 

Relate  to  general  knowledge 

Link  system  functions  to  general  knowledge  of  computer  capabilities 

Relate  to  previous  content 

Build  on  knowledge  and/or  skills  covered  earlier  in  the  course 

Respond  to  learners 

Provide  information  to  satisfy  student  questions  or  requests 

Encourage  active  learning 

Promote  student  involvement  by  means  of  instructor’s  challenges 

3  The  observation  form  was  revised  twice  based  on  the  results  of  pilot  tests  conducted  at  Ft.  Hood  and  Ft.  Benning. 
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Use  specific  dL  techniques 


Uses  instructional  techniques  specific  to  dL  environments 


Observers  recorded  the  activities  and  behaviors  in  five-minute  blocks  so  that  the 
observation  form  provided  a  chronology  of  the  training  activities.  Codes  were  used  by  the 
observers  so  that  the  activities  and  behaviors  could  be  recorded  quickly  (Table  4).  In  addition  to 
the  codes,  the  observers  provided  descriptions  of  the  activities  and  behaviors.  A  sample  of  a 
completed  form  appears  in  Table  5.  This  example  also  was  used  during  the  training  that  was 
provided  to  the  observers  prior  to  the  data  collection  to  ensure  consistency  in  the  coding  process 
which  is  further  described  in  the  Procedure  section. 


Table  4 

Codes  Used  to  Record  Instructor  Activities  and  Behaviors 


ACTIVITY  CODE 

VID  -  video 

DEM  -  demonstration 

G/D  -  guided  demo 

PE  -  practical  exercise 

REV  -  review 

BRK  -  break 

TEST  -  quiz,  exam,  etc 

LEC  -  lecture 

LEARNING  PRINCIPLES 

MA  -  uses  Memory  Aids 

RM  -  Relates  to  Military  operation 

EA-  Encourages  Active 
learning 

PP  -  Purpose  and  Path 

RP  -  Relates  to  Previous  content 

EP  -  Emphasizes  Practice 

RG  -  Relates  to  General 
knowledge 

SP-  points  out  unique  Screen 
Prompts  or  cues 

LC  -  Learning  Check 

RL  -  Responds  to  Learners 

dL  -  Uses  techniques  specific  to 

dL 

Table  5 

Sample  of  Completed  Part  II A  of  the  Instructor  Obseri’ation  Form 


Num 

Activity 

Code 

Time 

Description  of  topics  covered  and  instructor/student 
behaviors 

Training 

Technique 

1 

LEC 

0815 

Reviewed  what  they  would  cover  for  the  day  and  reminded 
them  of  upcoming  PE 

PP 

Told  students  to  power  up  systems,  answered  questions 
from  students 

RL 

2 

G/D 

0820 

Instructor  provided  steps  to  build  3D  maps.  Two 
students  asked  clarifying  questions. 

SP,  RL 

0825 

Cont  G/D  to  3D  maps.  Stopped  multiple  times  to  answer 
student  questions. 

SP,  RL 

Appears  to  be  a  problem  with  workstations.  No  AI  is 
present  so  instructor  is  troubleshooting. 

RL 

3 

DEM 

0830 

Nobody  can  build  3D  map  so  instructor  just  demonstrated. 
Explained  operational  uses. 

SP,  RM 

4 

G/D 

Began  a  G/D  to  locate  and  select  a  map.  Instructor  had  to 
help  some  students  properly  configure  their  machines. 

SP,  RL 

0835 

Continued  G/D  on  selecting  a  map 

SP 

Note.  Reproduced  from  Leibrecht  et  al.  (2007). 
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Following  Leibrecht  et  al.’s  (2007)  procedure,  the  observers  also  were  required  to 
describe  the  practical  exercises  that  were  coded  on  the  chronology  form.  As  shown  in  Appendix 
A,  the  observers  recorded  the  type  of  exercise,  the  exercise  context,  how  the  instructor  monitored 
progress,  whether  peer  coaching  was  observed,  the  number  of  repetitions  of  the  same  exercise, 
and  additional  explanatory  comments.  The  codes  for  the  type  of  exercises  reflected  the  extent  to 
which  the  practical  exercises  challenged  the  students.  For  example,  a  less  challenging  activity 
would  be  one  in  which  the  instructor  leads  the  students  through  the  procedures  while  a  more 
challenging  activity  would  be  one  in  which  the  students  were  required  to  apply  knowledge  or 
skills  learned  at  earlier  points  in  the  course.  The  codes  for  the  type  of  exercise  context  reflected 
whether  the  activities/tasks  were  conducted  within  a  simulated  military  operation,  whether  they 
were  job  relevant  but  not  necessarily  operational,  or  whether  they  were  performed  independent 
of  a  specific  context  (e.g.,  send  a  free  text  message).  Finally,  the  codes  for  how  the  instructor 
monitored  student  progress  included  whether  the  instructor  obtained  student  feedback  and 
observed  student  performance. 

Instructor  interview  protocol.  Interview  questions  were  developed  to  determine  the 
specific  instructional  design  and  learning  principles  that  the  instructors  considered  when 
preparing  to  teach/present  the  distributed  FBCB2  course  (Appendix  B).  The  interview  protocol 
also  included  questions  aimed  at  determining  effective  techniques  for  facilitating  learning, 
keeping  the  students’  on  the  same  pace,  and  answering  students’  questions  in  a  distributed 
classroom  environment.  Finally,  several  questions  tapped  the  instructors’  opinions  of  the 
advantages  and  disadvantages  of  teaching  digital  skills  in  a  distributed  environment,  problems 
they  typically  encounter  teaching  the  course,  their  likes  and  dislikes  in  teaching  the  course  in  a 
distributed  format,  and  suggestions  for  course  improvements. 

Student  observation  form.  We  modeled  the  student  observation  forms  after  the  instructor 
observation  packet  (Appendix  C).  Specifically,  the  observation  packet  consisted  of  three  forms: 
1)  Part  I:  basic  classroom  details  (e.g.,  number  of  students,  general  description  of  the  training 
site);  2)  Part  II:  a  chronology  of  instructor  activities  and  student  behaviors;  and  3)  Part  III: 
summary  questions  concerning  the  nature  of  the  skills  covered  during  that  observation  period, 
technical  problems  experienced  during  that  observation  period,  and  innovative  teaching 
techniques  that  were  noted. 

Similar  to  the  instructor  observation  form,  Part  II  used  the  same  activity  codes  to  more 
accurately  link  the  student  behaviors  to  specific  instructional  events  and,  as  shown  in  Table  6, 
included  codes  for  the  specific  student  behaviors  that  occurred  during  the  course.  The  codes 
reflected  the  range  of  possible  behaviors  that  could  occur  within  the  distributed  classroom 
environment  and,  as  with  the  instructor  codes,  were  based  on  the  belief  that  these  behaviors 
could  be  recorded  in  real-time  with  a  high  degree  of  reliability. 

Similar  to  the  instructor  observation  form,  the  observers  recorded  the  student  behaviors  in 
five-minute  blocks  to  provide  a  chronology  of  behavior.  In  addition  to  the  codes,  the  observers 
provided  descriptions  of  the  behaviors.  A  sample  of  a  completed  form  appears  in  Table  7.  This 
example  also  was  used  during  the  training  that  was  provided  to  the  observers  prior  to  the  data 
collection  to  ensure  consistency  in  the  coding  process  which  is  further  described  in  the 
Procedure  section. 
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Table  6 

Codes  Used  to  Record  Student  Behaviors 


Student  Behaviors 

OB  -  Other  student  Behaviors 
are  observed 

SM  -  Instructor  takes  over  Student’s 
Mouse 

AR  -  Asks  instructor  to  Repeat  step 

NP  -  Does  Not  Pay  attention  to 
the  lecture/demonstration 

UO  -  Uses  Other  functions  than  those 
for  current  task 

AQ  -  Asks  instructor  a  Question 

PA  -  Pays  attention  to 

lecture/PowerPoint 

presentation/demonstration 

WF  -  Waits  for  others  to  Finish 
task/PE 

GS  -  Works  in  Groups  to  Solve 
individual  tasks 

PC  -  Peer  Coaching  (seeks/gives 
help) 

WA  -  Works  Ahead  of  instruction  but 
on  same  task 

IS  -  Works  Independently  to  Solve 
problems 

RQ  -  Responds  to  instructor 

Questions 

LB  -  Lags  Behind  instruction 

SC  -  Side  Conversations  during 
instruction 

Table  7 

Sample  of  Completed  Part  II  of  the  Student  Observation  Form 


Num 

Activity 

Code 

Time 

Description  of  topics  covered  and  instructor/student 
behaviors 

Student 

Behaviors 

2 

DEM 

0820 

Steps  to  build  3D  maps.  Two  students  asked  clarifying  questions. 

AQ 

0825 

Con’t  Demo  3D  maps.  Student  works  ahead  of  instructor  and 
builds  his  own  3D  map. 

WA 

3 

G/D 

0830 

G/D  3D  maps.  Several  students  cannot  keep  up  with  the 
instructor. 

LB 

0835 

Con’t  G/D  3D  maps.  Instructor  takes  over  mouse  of  one  student  to 
show  him  the  steps  of  building  a  3D  map. 

SM 

Student  survey.  A  survey  was  developed  to  determine  the  students’  perceptions  of  the 
effects  of  the  distributed  classroom  environment  on  both  their  motivation  and  ability  to  learn 
digital  skills  (Appendix  D).  The  students  also  were  asked  to  provide  the  extent  to  which  they 
sought  assistance  from  their  peers,  their  reactions  to  the  pace  of  instruction,  any  problems  they 
encountered,  their  likes  and  dislikes  about  the  course,  and  recommendations  for  improvements 
The  survey  also  included  demographic  questions  to  determine  the  students’  experience  with 
digital  systems  and  computers. 
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Procedure 


Two  subject  matter  experts  (SMEs)  collected  the  data.  To  facilitate  consistency,  one 
SME  observed  instructors  and  the  other  SME  observed  students  for  both  courses.  Both  of  the 
observers  were  retired  Army  personnel  with  extensive  experience  using  ABCS  systems,  training 
digital  skills,  and  observing  classroom  training  in  Army  schools  and/or  training  centers.  Further, 
both  had  also  conducted  observations  for  the  residential  research  project,  Leibrecht  et  al.  (2007). 
For  the  present  research,  the  observers  observed  100%  of  both  the  instructor  and  student  class 
time. 


Prior  to  the  first  data  collection,  the  observers  participated  in  training  sessions  to  increase 
the  reliability  and  validity  of  the  data.  The  observers  were  provided  with  instructions  that 
explained  the  procedures  for  recording  the  activities  and  behaviors  (see  Appendix  E  for 
instructions  for  both  the  instructor  and  student  observations).  Then,  a  senior  investigator  led  the 
observers  through  the  observation  process,  discussing  procedural  consistency,  and  clarifying 
issues  raised  by  the  observers. 

A  standard  procedure  was  used  for  observing  the  instructors  and  student  classes.  Well 
before  each  course,  each  observer  coordinated  by  e-mail  and/or  phone  with  the  respective 
instructor  or  the  training  site  point  of  contact.  In  each  case,  this  coordination  included  an 
explanation  of  the  purpose  and  role  of  the  observation  and  gathering  course  information  aimed  at 
improving  the  observation  while  minimizing  the  impact  on  the  instructor  and  students.  On  the 
first  class  day,  observers  arrived  early  at  their  respective  sites  to  make  any  final  coordination  and 
for  face-to-face  introductions.  In  the  case  of  the  student  observer,  this  was  to  the  unit  chain  of 
command  and  included  an  explanation  of  the  purpose  and  role  of  the  observation  and  discussion 
of  any  concerns  they  may  have  had  regarding  the  data  collection. 

The  observers  captured  the  data  directly  on  the  observation  forms,  recording  all  of  the 
data  and  notes  by  hand.  As  can  be  seen  in  Tables  5  and  7,  the  observers  numbered  each  activity 
(i.e.,  lecture,  guided  demonstration,  practical  exercise,  etc.)  sequentially.  Using  a  time-sampling 
procedure,  the  observers  entered  the  code  for  each  activity  observed  during  a  5-min  block.  Then, 
they  described  the  content  that  was  covered  during  that  time  block,  along  with  noteworthy 
student  or  instructor  behaviors.  Finally,  they  entered  the  code  for  the  specific  training  technique 
or  student  behavior  that  was  observed  in  the  final  column  of  the  chronology  form.  It  is  important 
to  note  that  every  activity,  training  technique,  and  behavior  that  occurred  within  a  5-min  block 
was  only  counted  once;  additional  occurrences  in  the  same  block  were  ignored.  The  instructor 
observer  was  collocated  with  the  instructor  in  the  “Broadcast  Booth”  (see  description  below)  and 
took  precautions  to  avoid  interfering  with  the  instruction.  The  student  observer  was  located 
inconspicuously  in  the  classroom  as  to  not  affect  the  students’  behavior. 

The  instructor  observer  conducted  individual  interviews  with  the  instructors  at  a  time 
near  the  end  of  the  course,  as  previously  coordinated  with  the  instructors.  In  each  case,  the 
instructor  interview  protocol  was  followed,  Appendix  B.  Where  appropriate,  the  follow-on 
questions  and  prompts  included  in  the  primary  questions  were  used  to  elicit  more  detailed 
information.  The  interviews  varied  in  length  but  were  less  than  an  hour  in  duration  and  were 
tape  recorded  to  allow  for  augmentation  of  the  observer’s  interview  notes. 
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With  prior  coordination  by  the  observer  with  the  instructor  and  the  students,  the  student 
surveys.  Appendix  D,  were  administered  to  the  students  after  their  completion  of  the  course  final 
exam.  Since  test  completion  times  varied  widely,  students  completed  the  survey  individually 
with  the  observer  present  to  resolve  any  questions.  All  responses  were  anonymous;  the 
biographical  responses  did  not  include  any  identifying  information. 
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RESULTS 


Instructor  Workstation 

As  a  goal  of  the  course  is  to  provide  distributed  FBCB2  instruction,  the  instructors  were 
remotely  located  from  the  students  and  interacted  with  them  by  means  of  networked  video  and 
audio  capabilities.  The  course  uses  a  software  program  that  emulates  the  FBCB2  system,  and  the 
BCTC  has  over  900  supportable  server  connections.  The  instructors  conducted  the  dL  FBCB2 
training  from  “Broadcast  Booths”  located  at  Camp  Dodge,  IA.  These  “Booths”  were  small 
rooms  (approximately  8  by  10  feet)  that  contained  the  instructors’  stations  with  controls  which 
allowed  all  instruction  to  be  conducted  remotely.  The  instructors  sat  in  front  of  a  bank  of 
computer  screens  which  displayed  each  student’s  desktop  (up  to  32  screens  at  one  time).  There 
were  cameras  mounted  on  top  of  the  suite  of  monitors  which  displayed  the  instructor  to  the 
students  and  there  was  a  microphone  so  the  instructor  could  speak  with  the  students.  Thus,  the 
instructors  had  the  capability  of  presenting  PowerPoint  slides  to  their  students  as  well  as 
communicating  to  individual  students  or  to  the  entire  class  (see  below  for  specific  details). 

One  important  feature  of  the  Booths  was  the  instructor’s  ability  to  directly  monitor  and 
control  the  FBCB2  system  of  each  student  individually.  This  enabled  the  instructor  to  view 
students’  input  or  omissions  as  they  operated  their  systems.  As  necessary,  the  instructor  could 
take  control  of  a  student’s  mouse  and  directly  manipulate  the  input  to  his  system  as  a  means  of 
providing  individual  assistance.  Interconnectivity  between  the  instructor  and  student 
workstations  is  described  below,  see  Instructor- Student  dL  Networking. 

Student  Classroom  Environment 

The  students  occupied  three  separate  classrooms  during  the  two  courses.  In  most  respects 
each  student  classroom  resembled  a  resident  classroom.  In  fact,  all  were  dual-purpose,  designed 
for  residential  training  (or  office  space)  and  later  equipped  for  dL  instruction.  Each  classroom 
contained  rows  of  student  workstations  with  each  student  assigned  to  a  workstation  (terminal) 
that  connected  to  a  tactical  FBCB2  system  as  described  below. 

The  student  classrooms  contained  capabilities  that  were  unique  to  the  dL  method  of 
instruction.  To  enable  the  students  to  interact  with  the  remote  instructor,  the  classrooms 
contained  VTT  equipment.  The  VTT  components  included: 

•  Wheeled  cart  for  the  VTT  equipment. 

•  Large  screen  on  which  the  students  viewed  the  instructor,  the  PowerPoint  slides,  or  the 

FBCB2  display  being  used  for  demonstration  (instructor’s  or  student’s).  The  instructor 
controlled  the  view. 

•  Camera  via  which  the  instructor  observed  the  class  as  a  whole.  The  camera  afforded  no 

capability  to  view  a  single  student,  although  the  instructor  could  view  each  student’s 
FBCB2  display  on  a  separate  bank  of  monitors. 

•  Push-to-talk  microphones  enabled  voice  communication  between  the  classroom  and 

instructor.  Although  numbers  varied  by  classroom,  there  were  sufficient  microphones 
for  all  students. 
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•  At  least  two  speakers  (more  in  the  larger  classrooms)  facilitated  at-large  broadcasting  of 
voice  communications  from  both  the  instructor  and  students. 

During  one  course,  the  students  were  divided  between  two  classrooms.  A  camera  in  each 
classroom  enabled  the  instructor  to  view  the  students  in  either  room,  one  classroom  at  a  time. 

The  instructor’s  monitor  displayed  the  classroom  view  linked  to  the  most  recent  push-to-talk 
activity.  If  a  student  in  the  classroom  containing  the  “inactive”  camera  activated  a  push-to-talk 
button,  the  camera  view  switched  to  that  classroom.  If  the  instructor  wanted  to  switch  the  view 
from  the  current  classroom  to  the  other,  he  asked  for  someone  in  the  other  classroom  to  activate  a 
push-to-talk  button.  Each  classroom  could  not  view  the  other  classroom.  Thus,  there  was  no 
direct  interaction  between  the  two  classrooms,  but  all  students  could  listen  to  the  voice 
communications. 

Instructor-Student  dL  Networking 

Interaction  between  the  classroom(s)  and  the  instructor’s  station  was  accomplished  by 
using  two  separate  network  systems. 

•  The  VTT  connection  was  implemented  over  an  Army  National  Guard  Bureau  wide 
area  network  dedicated  to  Guard  VTT  use  and  not  part  of  the  commercial  Internet 
system. 

•  The  Battle  Command  Training  and  Distributed  System  (BCT&DS)  supported  FBCB2 
connectivity.  The  BCT&DS  allows  student-Soldiers  to  remotely  control  software  that 
emulates  a  FBCB2  system  housed  within  a  data  center  at  BCTC-Dodge.  Soldiers  use 
input  devices  (keyboard,  mouse)  and  video  access  via  a  secure  tunnel  over  the 
commercial  Internet  to  control  the  remote  systems.  The  FBCB2  systems  operate  on  a 
closed  local  area  network  within  the  BCT&DS,  with  inter-system  connectivity. 

When  the  bandwidth  of  either  network  system  became  overtaxed  due  to  shared  use,  some 
latency  was  experienced.  Bandwidth  limitations  typically  delayed  transmissions  between 
FBCB2  systems  (inter-site)  and  occasionally  degraded  the  video/audio  environment.  In  addition, 
the  failures  of  the  Internet  connection  or  the  VTT  equipment  sometimes  caused  interruptions  in 
the  instruction.  When  failures  occurred,  technicians  on  both  ends  worked  to  restore  connectivity. 
During  these  interruptions  the  students  typically  took  a  break. 

Instructor-to-Student  Ratio 

In  both  courses,  one  instructor  taught  each  class  from  start  to  finish  (Table  8).  No 
assistant  instructors  (AIs)  or  proctors  participated  in  either  course.  In  both  classes,  the  senior 
ranking  individual  performed  minimal  administrative  duties  on  an  informal  basis. 

As  Table  8  shows,  the  instructor-to- student  ratios  for  the  dF  courses  were  1:11  and  1:8, 
with  the  determining  factor  being  the  number  of  students  enrolled.  It  is  not  known  if  there  was  a 
limit  on  the  class  size,  but  the  student  classrooms  contained  no  more  than  17  workstations.  The 
instructor-to-student  ratios  in  the  distributed  courses  were  similar  to  those  reported  by  Feibrecht 
et  al.  (2007)  for  traditional  courses,  as  shown  in  Table  8. 
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Table  8 

Ratio  of  Instructors  to  Students  in  FBCB2  Courses  Obserx’ed 


Course 

#  Instructors 

#  Students 

Instructor-to- 
Student  Ratio 

Distributed  Courses 

FBCB2-1 

1 

ir 

1:11 

FBCB2-2 

1 

8b 

1:8 

Traditional  Courses  (from  Leibrecht  et  al.,  2007) 

FBCB2-1 

2 

19 

1:9.5 

FBCB2-2 

2 

19 

1:9.5 

a  One  student  dropped  out  after  Day  3  due  to  illness. 
b  Two  students  were  eventually  dropped  due  to  absences. 


Course  Objectives  and  Characteristics 

The  BCTC  courses  were  designed  to  take  National  Guard  Soldiers  with  basic  computer 
knowledge  and  experience  to  the  point  where  they  knew  how  to  perform  the  basic  functions  of 
an  FBCB2  workstation.  Thus,  the  lessons  were  organized  around  the  functional  features  of  the 
system  (preparation  for  combat,  messaging,  navigation,  orders,  overlays,  data  transfer,  and 
system  security).  Both  courses  observed  were  five  days  in  duration  (Monday  -  Friday).  This 
included  class  start-up,  primary  instruction,  testing,  and  course  wrap-up.  The  student  observer 
estimated  that  both  courses  focused  100%  on  operating  the  user  interface  (“knobology”)  during 
the  instructional  phase  (Days  1-4),  and  100%  on  applying  the  system  capabilities 
(“employment”)  during  the  final  exam  (Day  5). 

The  sequence  of  instruction  followed  a  general-to-specific  approach.  Instruction  began 
with  a  system  overview  of  the  FBCB2  platform.  System  description  and  initialization  came  next, 
followed  by  readying  the  system  for  operations.  An  orientation  to  the  system’s  main  screen  was 
typically  followed  by  a  series  of  learning  activities  organized  around  the  layout  and  structure  of 
system  features  and  functions.  This  led  to  a  drill-down  cascade  as  the  instructor  followed  menu 
options  to  deeper  levels  of  functioning,  then  returned  to  a  higher  level  to  move  to  the  next  major 
function  or  feature.  The  instructor  used  performance  challenges  and  practical  exercises  to 
integrate  and  consolidate  knowledge  and  skills  across  major  functions.  Each  course  ended  with  a 
capstone  (final)  exam,  preceded  by  a  review  of  cumulative  topics. 
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Instructional  Activities  for  clL  Classes 


This  subsection  presents  the  results  regarding  the  instructor’s  teaching  process  (basic 
activities,  techniques  employed,  etc.).  The  following  topics  organize  the  presentation: 

•  Basic  Activities 

•  Training  Techniques 

•  Training  Aids 

•  Practical  Exercises 

•  Testing 

Basic  activities.  As  explained  in  the  Method  section,  the  set  of  eight  instructional 
activities  included  conventional  methods  ranging  from  lecturing  by  the  instructor  to 
demonstration  (with  and  without  student  participation)  to  testing  by  means  of  quizzes  and 
examinations.  These  activities  formed  the  categories  used  to  classify  instructor  behaviors,  and 
this  subsection  describes  their  frequency  of  occurrence. 

The  methodology  called  for  the  observer  to  record  every  activity  that  occurred  at  least 
once  in  a  5-min  block,  using  the  pre-defined  codes.  In  the  vast  majority  of  cases  the  observer 
recorded  a  single  activity  code  per  time  block.  Occasionally  he  recorded  two  or  even  three  codes 
in  a  5-min  block. 

Table  9  summarizes  key  parameters  for  both  courses.  The  second  column  of  the  table 
gives  the  total  number  of  5-min  blocks  observed  during  each  day,  excluding  breaks.  Within  each 
main  cell  appear  the  number  of  blocks  in  which  an  activity  was  recorded  and  the  same  value  (in 
parentheses)  expressed  as  a  proportion  of  the  total  blocks  observed  for  the  day.  Because  a  single 
block  could  contribute  multiple  counts  (one  for  each  type  of  activity  observed),  the  proportions 
in  Table  9  (counts  per  block)  typically  sum  greater  than  1.0  across  a  row. 

The  relative  frequencies  of  the  various  activities  were  highly  consistent  across  both 
courses.  By  a  substantial  margin,  the  most  frequent  activity  was  guided  demonstration.  This 
activity  took  place  in  44%  or  more  of  the  blocks  during  7  of  the  10  days  observed.  Test  activities 
were  the  next  most  frequent,  occurring  in  nearly  one-fourth  of  the  blocks  overall  (but  heavily 
concentrated  in  the  last  day  of  the  course).  Lecture  was  frequent  during  the  first  day  of  each 
course  (occurring  in  35-44%  of  the  blocks),  but  it  dropped  off  sharply  in  subsequent  days. 
Practical  exercises  accounted  for  20-40%  of  the  blocks  during  5  of  the  10  days  observed.  In 
three  of  the  days,  practical  exercises  did  not  occur.  Review  of  previous  materials  took  place  in 
14%  of  the  blocks  overall,  and  was  rare  (10%  or  less)  or  absent  in  5  of  the  10  days  observed. 

Two  activities  were  notable  because  of  their  overall  absence  during  the  observed  classes. 
A  demonstration  without  students  participating  was  observed  in  only  two  days  (nine  blocks  of 
time)  of  one  course.  A  video  presentation  occurred  in  only  one  course,  appearing  merely  once 
and  involving  only  three  blocks.  The  video  illustrated  FBCB2  capabilities  employed  in  combat. 
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The  last  row  of  Table  9  sums  the  data  across  both  courses  to  shed  light  on  overall  trends. 
In  descending  order  from  most  frequent  to  least,  the  summed  data  show  the  following  hierarchy 
of  instructional  activities:  guided  demonstration,  testing,  practical  exercise,  review,  and  lecture, 
with  instructor-only  demonstrations  and  videos  nearly  absent. 


Table  9 

Frequency  of  Instructional  Activities 


Day 

Total 

Blocks 

Lecture 

Video 

Demo 

Guided 

Demo 

Practical 

Exercise 

Review 

Test 

FBCB2  Distributed  Course  #1 

1 

68 

24  (.35) 

0 

0 

30  (.44) 

13  (.19) 

5  (.07) 

0 

2 

67 

1  (-01) 

3  (.04) 

0 

42  (.63) 

0 

7  (.10) 

17  (.25) 

3 

74 

0 

0 

3  (.04) 

44  (.59) 

25  (.34) 

16  (.22) 

0 

4 

64 

2  (.03) 

0 

6  (.09) 

32  (.50) 

25  (.39) 

5  (.08) 

0 

5 

67 

0 

0 

0 

0 

0 

10  (.15) 

57  (.85) 

1-5 

340 

27  (.08) 

3  (.01) 

9  (.03) 

148  (.44) 

63  (.18) 

43  (.12) 

74  (.22) 

FBCB2  Distributed  Course  #2 

1 

36 

16  (.44) 

0 

0 

17  (.47) 

5  (.14) 

0 

0 

2 

71 

2  (.03) 

0 

0 

42  (.59) 

10  (.14) 

26  (.37) 

0 

3 

80 

0 

0 

0 

24  (.30) 

27  (.34) 

11  (.14) 

18  (.24) 

4 

60 

3  (.05) 

0 

0 

29  (.48) 

11  (.18) 

9  (.15) 

14  (.23) 

5 

36 

0 

0 

0 

0 

0 

0 

36(1.0) 

1-5 

283 

21  (.07) 

0 

0 

1 12  (.40) 

53  (.19) 

46  (.16) 

68  (.24) 

All  Courses/Days  (Summed) 

All 

623 

48  (.08) 

3  (.005) 

9  (.01) 

260  (.42) 

116  (.19) 

89  (.14) 

142  (.23) 
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Training  techniques  ancl  training  aids.  As  described  in  the  Method  section,  eleven 
training  techniques  provided  the  framework  for  characterizing  the  nature  of  the  classroom 
teaching  methods.  The  techniques  were  intended  to  reveal  how  the  instructor  applied  various 
learning  principles.  Guided  by  the  observation  form,  the  observer  recorded  codes  for  the 
techniques  that  occurred  in  each  5-min  block  (one  code  for  each  technique  observed  one  or  more 
times).  The  tally  process  excluded  breaks.  This  subsection  presents  both  quantitative  and 
qualitative  data  for  the  training  techniques. 

The  training  technique  tallies  for  both  courses  (day  by  day)  appear  in  Table  11.  Each  of 
the  table’s  data  cells  gives  a  frequency  count  and  rate  (count  per  block).  The  rate  of  occurrence 
was  computed  by  dividing  the  frequency  count  by  the  number  of  valid  5-min  blocks  for  the  day. 
The  valid  blocks  excluded  breaks  and  cases  of  missing  data.  Because  single  blocks  often 
contributed  multiple  counts  (one  for  each  different  technique  observed),  the  proportions  in  Table 
10  (counts  per  block)  sum  greater  than  1.0  across  a  row. 

In  both  courses  overall,  the  most  common  training  techniques  were  using  dL-specific 
techniques  and  checking  the  progress  of  student  learning,  occurring  in  about  half  the  total  blocks. 
They  were  followed  closely  by  relating  materials  to  general  knowledge,  observed  in  almost  half 
the  blocks.  Day  by  day  these  three  techniques  accounted  for  30-92%  of  the  total  blocks, 
excluding  the  final  exam  (Day  5).  Predictably,  learning  checks  (testing)  accounted  for  85-100% 
of  the  total  blocks  on  the  day  of  the  final  exam.  Relating  the  instruction  to  military  operations 
and  previous  content  occurred  with  low  frequency  (in  12%  and  8%  of  the  blocks,  respectively), 
as  did  responding  to  learners  (in  8%  of  the  blocks).  Pointing  to  prompts  or  cues  on  a  publicly 
viewed  display  and  encouraging  active  learning  were  observed  very  infrequently,  accounting  for 
only  2%  of  the  blocks. 

In  each  course,  three  training  techniques  appeared  rarely  (once  per  100  blocks  or  less,  on 
the  average),  emphasizing  practice  and  giving  purpose  and  path,  or  never,  using  memory  aids. 
When  the  instructors  stated  the  purpose  and  path  for  a  particular  FBCB2  function,  they  generally 
provided  the  purpose  (why  and  when  to  use  the  function  in  combat)  before  covering  the  path 
(i.e.,  actual  operating  procedures).  For  example,  the  instructor  might  explain  that  pre-setting  the 
Quick  Send  button  for  a  frequent  report,  such  as  a  Spot  Report,  will  save  critical  time  in  the  heat 
of  battle.  He  would  then  lead  the  students  through  the  steps  for  assigning  a  report  to  the  Quick 
Send  button. 

The  training  technique  labeled  “Use  dF  techniques”  was  actually  an  umbrella  category 
for  instructor  actions  specific  to  the  dF  environment.  The  following  examples  illustrate  the 
instructor  actions  that  fell  in  this  category: 

•  Monitor  students’  FBCB2  inputs  via  a  bank  of  displays. 

•  Query  a  specific  student  via  microphone  to  determine  if  he  needed  help. 

•  Verbally  coach  a  student  via  at-large  speaker. 

•  Take  control  of  a  student’s  mouse  to  demonstrate  or  rectify  process  steps. 

•  Send  administrative  announcements  via  FBCB2. 

•  Switch  the  instructor’s  classroom  view  (by  asking  a  student  to  key  his  microphone). 

•  Monitor  the  health  of  the  Internet  connectivity. 

•  Adjust  to  a  degraded  video  or  audio  stream. 
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Table  10 

Frequency  of  Observed  Training  Techniques 


Training  Techniques 

Day 

Point  to 
Screen 
Prompts 

Emphasize 

Practice 

Use 

Memory 

Aids 

Check 

Learning 

Progress 

Give 
Purpose 
&  Path 

Relate  to 
Military 
Ops 

Relate  to 
General 
Knowledge 

Relate  to 
Previous 
Content 

Respond 

to 

Learners 

Encourage 

Active 

Learning 

Use  dL 
Techniques 

FBCB2  Distributed  Course  #1 

1 

5  (.07) 

0 

0 

21  (.31) 

1  (-01) 

12  (.18) 

44  (.66) 

5  (.07) 

2  (.03) 

1  (-01) 

30  (.45) 

2 

0 

0 

0 

26  (.40) 

1  (-02) 

12  (.18) 

37  (.57) 

3  (.05) 

2  (.03) 

0 

44  (.68) 

3 

0 

1  (.01) 

0 

25  (.37) 

0 

20  (.30) 

33  (.49) 

5  (.07) 

20  (.30) 

9  (.13) 

53  (.79) 

4 

5  (.08) 

0 

0 

29  (.45) 

0 

12  (.19) 

32  (.50) 

6  (.09) 

0 

5  (.08) 

40  (.62) 

5 

0 

0 

0 

56  (.85) 

0 

0 

0 

10  (.15) 

0 

0 

0 

1-5 

10  (.03) 

1  (.003) 

0 

157  (.48) 

2  (.006) 

56  (.17) 

146  (.44) 

29  (.09) 

24  (.07) 

15  (.05) 

167  (.51) 

FBCB2  Distributed  Course  #2 

1 

0 

0 

0 

11  (.30) 

3  (.08) 

2  (.05) 

34  (.92) 

0 

1  (.03) 

0 

34  (.92) 

2 

0 

1  (-01) 

0 

35  (.49) 

0 

15  (.21) 

41  (.58) 

0 

5  (.07) 

0 

42  (.59) 

3 

0 

3  (.04) 

0 

35  (.44) 

0 

3  (.04) 

24  (.30) 

18  (.22) 

4  (.05) 

0 

26  (.32) 

4 

0 

0 

0 

25  (.42) 

0 

0 

29  (.48) 

0 

17  (.28) 

0 

30  (.50) 

5 

0 

0 

0 

36  (1.0) 

0 

0 

0 

0 

1  (.03) 

0 

1  (.03) 

1-5 

0 

4  (.01) 

0 

142  (.50) 

3  (.01) 

20  (.07) 

128  (.45) 

18  (.06) 

28  (.10) 

0 

133  (.47) 

All  Courses/Days  (Summed) 

All 

10  (.02) 

5  (.008) 

0 

299  (.49) 

5  (.008) 

76  (.12) 

274  (.45) 

47  (.08) 

52  (.08) 

15  (.02) 

300  (.49) 

Training  aids.  The  instructors  used  a  variety  of  training  aids  as  they  executed  the 
instructional  activities.  These  training  aids  included  traditional  materials  as  well  as  hardware  and 
software  unique  to  the  dL  environment. 

Based  on  traditional  classroom  methods,  a  package  of  instructional  slides  was  a  key  part 
of  the  program  of  instruction  (POI).  Conveying  both  text  and  imagery,  the  slides  were  projected 
on  a  large  screen  for  the  entire  class  to  view  simultaneously.  Various  static  images  of  the  user 
interface  (screen  captures)  were  displayed  frequently  in  the  slides.  Video  materials  (e.g.,  scenes 
from  tactical  operations)  were  displayed  on  only  one  occasion. 

Spiral  bound  Student  Handouts  were  shipped  by  the  schoolhouse  to  the  classroom  site 
and  issued  to  the  students  at  the  start  of  the  course  by  a  member  of  the  local  staff  administering 
the  dL  classroom.  Serving  as  a  guide  and  repository  of  essential  information,  the  handout 
contained: 

•  Training  schedule, 

•  PowerPoint  slides  printed  two  per  page  (297  used  during  the  course), 

•  Tactical  materials  (task  organization  and  maps  for  scenarios), 

•  Practical  exercises  (all  nine  used  during  the  course), 

•  Final  examination  (hardcopy  form), 

•  FBCB2  references  (techniques  and  procedures  handbook,  operator’s  pocket  guide),  and 

•  FBCB2  sustainment  training  package  on  compact  disc. 

The  handout  was  used  by  the  instructor  and  students  throughout  the  course  as  a  reference 
and  source  of  course  materials.  However,  it  was  difficult  to  use  because  (a)  some  out-of-date 
procedures  did  not  match  the  current  FBCB2  software  and  (b)  lack  of  continuous  page  numbers 
made  it  hard  to  locate  materials  referenced  by  the  instructor. 

The  instructors  could  display  an  active  FBCB2  user  interface  on  a  large  screen  display. 
The  system  thus  displayed  belonged  to  either  the  instructor  or  a  “student  driver.”  The  instructors 
used  the  “live  screen”  view  in  performing  guided  demonstrations  of  the  user  interface  features, 
operating  procedures,  or  desired  system  end-states  and  outcomes. 

Because  the  instructor’s  actions  on  his  FBCB2  experienced  transmission  delays  before 
arriving  in  the  classroom  (“lag  time”),  a  “student  driver”  was  often  used  for  demonstrations.  The 
student  driver  was  an  instructor-selected  student  with  competent  computer  skills  and  good 
knowledge  of  FBCB2  (or  a  quick  learner  in  the  absence  of  prior  experience).  The  student 
driver’s  screen  was  displayed  on  the  large  screen  for  the  other  students  to  see. 

The  bank  of  monitors  displaying  the  FBCB2  screens  of  individual  students  in  the  Booth 
provided  essential  information  for  monitoring  the  progress  of  learning.  This  capability  was 
especially  valuable  for  identifying  when  a  student  was  struggling  with  a  task  or  skill. 

Finally,  through  a  specialized  software  tool,  the  instructors  were  able  to  take  control  of  a 
student’s  mouse.  This  was  typically  employed  when  a  student  was  having  considerable 
difficulty  which  might  slow  the  entire  class  or  affect  the  individual  student’s  learning  process. 
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Summary  of  innovative  training  techniques  and  training  aids.  The  observer  noted  that 
the  following  examples  of  the  techniques  and  aids  described  above  were  particularly  effective  for 
distributed  classroom  environments. 

•  In  both  courses  the  instructors  selected  one  of  the  more  capable  students  to  serve  as  a 
demonstrator  (“student  driver”),  partly  to  avoid  long-haul  transmission  lag. 

•  Peer  coaching  occurred  extensively,  with  encouragement  from  the  instructors  to  help 
offset  their  physical  separation  and  lack  of  an  AI. 

•  A  routine  dL  mechanism  involved  the  instructor  taking  control  of  a  student’s  mouse 
to  ensure  proper  execution  of  procedural  steps. 

•  “Mini-exercises”  (performance  challenges)  were  employed  to  reinforce  key  skills  or 
tasks  and  enable  the  instructors  to  perform  structured  learning  checks. 

•  At  the  end  of  one  practical  exercise,  the  instructor  asked  a  student  to  walk  him 
through  a  specific  set  of  steps. 

•  The  student  handout  included  a  sustainment  training  package  on  compact  disc  that 
Soldiers  could  use  to  practice  on  their  own. 

Practiced  Exercises 

Summary  data  for  the  practical  exercises  observed  in  both  courses  appear  in  Table  11. 
Integration  of  prior  tasks  was  the  predominant  type  of  exercise  (13  of  14  cases).  Guided  exercise 
accounted  for  the  remaining  case.  Repeating  a  demonstration  and  utilizing  a  new  situation  did 
not  occur.  This  pattern  is  different  from  that  seen  previously  in  traditional  courses,  where  60% 
of  the  definitive  cases  involved  integration  of  prior  tasks  and  repeating  a  demonstration  was  the 
only  type  not  observed  (Leibrecht  et  al.,  2007). 
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Table  11 

Practical  Exercises  Obserx’ed  in  Distributed  and  Traditional  Courses 


Course 

Total 
#  PEs 

Number  of  PEs  per  Type 

Guided 

Exercise 

Repeat 

Demo 

New 

Situation 

Integration  of 
Prior  Tasks 

Not 

Recorded'' 

Distributed 

Courses 

FBCB2-1 

9 

1 

0 

0 

8 

— 

FBCB2-2 

5 

0 

0 

0 

5 

— 

Total 

14 

1 

0 

0 

13 

— 

Traditional 

Courses'3 

FBCB2-1 

4 

1 

0 

1 

2 

- 

FBCB2-2 

3 

0 

0 

0 

1 

2 

Total 

7 

1 

0 

1 

3 

2 

a  Type  was  not  recorded  by  the  observer. 

b  Data  for  traditional  courses  are  taken  from  Leibrecht  et  al.  (2007). 


Practical  exercises  in  the  distributed  courses  were  set  in  a  job-relevant  context  (57%  of 
the  cases)  or  military  operations  setting  (43%  of  the  cases).  An  arbitrary  context  (e.g.,  send  a 
free  text  message)  was  not  used  during  practical  exercises.  In  contrast,  all  of  the  definitive  cases 
in  the  traditional  courses  involved  a  job-relevant  context  (Leibrecht  et  al.,  2007). 

In  the  distributed  courses,  the  instructor  nearly  always  monitored  student  progress  and 
problems  by  directly  observing  them  at  work  (93%  of  the  cases).  Querying  students  to  obtain 
feedback  occurred  in  only  one  exercise  (7%  of  the  cases).  Peer  coaching  was  very  common, 
occurring  in  86%  of  the  cases.  This  pattern  is  quite  similar  to  what  was  seen  in  the  previous 
research  project,  where  monitoring  of  students  always  entailed  observing  them  and  peer 
coaching  took  place  in  60%  of  the  definitive  cases  (Leibrecht  et  al.,  2007). 

Aided  by  their  VTT  capabilities,  the  instructors  sometimes  assisted  students  having 
difficulties  during  practical  exercises.  Instructor  assistance  occurred  in  at  least  4  of  the  14 
exercises.  More  common  was  the  peer  coaching  mentioned  in  the  preceding  paragraph. 

A  practical  exercise  consumed  from  2  to  21  5-min  blocks.  An  exercise  usually  involved 
4  or  more  5-min  blocks — about  20  min  or  more  (86%  of  the  cases).  In  57%  of  the  cases,  the 
duration  exceeded  6  blocks  (about  30  min),  surpassing  100  min  in  one  case.  In  the  traditional 
courses  (Leibrecht  et  al.,  2007),  the  range  of  exercise  durations  was  similar  but  the  number  of 
blocks  exceeded  6  in  only  one  case  (14%  of  the  exercises). 

In  both  dL  courses,  the  faster  students  frequently  finished  the  practical  exercises  early 
and  were  allowed  to  take  a  break.  The  early  finishers  sometimes  opted  to  assist  fellow  students. 
Both  of  these  conditions  also  occurred  in  the  traditional  courses  (Leibrecht  et  al.,  2007). 
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Testing 


Quizzes  or  tests  took  place  occasionally  in  both  courses.  On  Day  2  of  FBCB2-1,  six  tests 
involved  1-4  time  blocks.  In  FBCB2-2,  one  test  occurred  on  Day  3  (lasting  for  18  blocks)  and 
three  tests  (3-6  blocks  in  duration)  happened  on  Day  4.  The  tests  were  labeled  scenario  exercises 
or  reinforcement  exercises,  with  scenario-based  test  materials  displayed  in  PowerPoint  slides  on 
the  screen  and/or  (in  FBCB2-2)  placed  in  the  Student  Handout.  Coaching  by  the  instructor  and 
peers  was  noted  on  occasion  during  tests.  In  no  case  was  post-test  review  or  feedback  recorded 
by  the  observer.  However,  the  instructors  worked  with  the  students,  as  necessary,  until  everyone 
successfully  completed  a  given  test. 

Both  distributed  courses  included  a  final  exam  as  a  capstone  event  (Table  12).  Organized 
as  a  practical  exercise,  the  final  exam  strongly  emphasized  hands-on  tasks  and  included  minimal 
fill-in  items.  In  both  courses  the  final  exam  allowed  students  to  work  with  all  materials  in  the 
Student  Handout  and  their  class  notes.  Completion  time  was  targeted  for  three  hours,  but  actual 
completion  took  four  to  five  hours.  Grading  of  the  exams  relied  on  Go/No  Go  scoring. 

Table  12 

Summary  of  Final  Exams  Used  in  Distributed  and  Traditional  Courses 


Courses 

Written  Component 

Hands-on  Component 

Estimated 

Time 

Distributed 

2  fill-in  (recall)  questions 
scored  Go/No  Go 

30  discrete  tasks  organized  in  three  phases 
(outcomes  scored  Go/No  Go) 

3  hrs 

Traditional1 

6  fill-in  (recall)  questions  scored 
Go/No  Go 

34  discrete  tasks  with  outcomes  scored 

Go/No  Go 

3  hrs 

a  Data  for  traditional  courses  are  taken  from  Leibrecht  et  al.  (2007). 
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The  instructor ’s  view.  The  instructors  used  a  Guided  Experiential  Leaming-like  method 
to  deliver  step-wise  instruction  -  demonstration,  practice,  evaluation.  During  interviews  both 
instructors  stated  that,  based  on  their  experience  with  dL  instruction,  a  blend  of  techniques  is 
essential.  The  techniques  they  believe  are  most  effective  in  the  distributed  environment,  and  a 
description  of  their  practice  in  the  course,  are  summarized  in  Table  13. 

Table  13 

Effective  clL  Instruction  Techniques  (Instructor  View) 


Technique 

Description 

Leveraging  Experience 

Instructor  relates  topics  to  students’  previous  experiences  if 
they’ve  used  the  system  before. 

Guided  Demonstration 

Instructor  performs  the  steps  of  a  skill  while  students  observe  and 
mimic,  each  using  his  assigned  FBCB2  workstation. 

Hands-on  Practice 

Guided  demonstration  is  followed  promptly  by  individual  practice 
until  students  are  familiar  with  the  skill’s  steps.  Students  who  are 
having  difficulty  learning  the  steps  receive  extra  scenarios. 

Consistent  Scenarios 

Linked  scenarios  are  used  throughout  the  course  to  help  students 
understand  operating  procedures  and  their  application  to  tactical 
operations. 

Student  Monitoring 

Instructor  observes  progress  of  each  student  by  monitoring  actions 
via  continuous  view  of  each  student’s  workstation.  Directs 
questions  to  students  who  are  not  paying  attention. 

Learning  Checks 

“Mini-PEs”  are  used  to  assess  student  understanding  of  a  limited 
aspect  of  the  current  topic. 

Student  Driver 

A  knowledgeable  student  serves  as  a  surrogate  demonstrator  to 
overcome  transmission  lag  of  the  instructor’s  FBCB2. 

The  instructors  explained  that  their  fundamental  approach  was  to  “walk  them  through  the 
steps”  (guided  demonstration)  and  then  have  students  practice  the  procedure  on  their  assigned 
workstations,  usually  reinforced  with  a  PE.  The  ability  to  use  the  individual  FBCB2  monitors  to 
observe  specific  actions  on  each  student’s  interface  allowed  the  instructors  to  closely  monitor 
each  student’s  progress  during  the  hands-on  practice  and  PEs.  This  enabled  one-on-one 
coaching  as  well  as  adjustment  of  the  pace  of  instruction  based  on  the  students’  performance. 
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Comparison  to  Traditional  Instruction 

Due  to  slight  differences  in  the  POIs  for  the  traditional  and  dL  courses  as  well  as 
differences  in  the  observation  procedures  used  in  the  predecessor  research  project,  we  compared 
the  instructional  activities  and  training  techniques  for  specific  topics.4  Specifically,  we  examined 
the  activities  and  techniques  for  messages  and  creating  and  sending  orders  and  overlays.  These 
topics  were  chosen  because  a  significant  amount  of  instructional  time  was  spent  on  them.  Tables 
14  and  15  break  out  the  data  for  instructional  activities  and  training  techniques,  respectively. 

The  topics  are  presented  both  separately  and  combined.  It  is  important  to  note  that  only  one 
traditional  class  was  observed  for  each  topic  -  so  these  percentages  are  based  on  lower  numbers 
compared  to  the  dL  courses. 

As  shown  in  Table  14,  the  dL  instructors  were  consistent  in  the  types  of  instructional 
activities  they  used  for  both  topics,  using  mainly  guided  demonstration  and  then  practical 
exercises,  review  and  testing.  However,  for  the  traditional  classes,  one  instructor  team  (the 
traditional  classes  had  AIs  whereas  the  dL  classes  did  not)  predominately  used  guided 
demonstration  for  messages  while  the  other  instructor  team  used  a  mix  of  techniques  similar  to 
the  dL  classes. 

When  combined  across  topics,  Table  14  shows  strong  similarities  between  the  dL  and 
traditional  classes.  Overall,  the  dL  instructors  spent  much  more  time  (judging  from  the  total 
number  of  blocks)  covering  both  of  the  selected  topics  than  the  traditional  instructors.  The 
greater  time  spent  reviewing  and  testing  students  accounts  for  some  of  the  additional  time  in  the 
dL  courses. 


4  The  traditional  classes  followed  a  6-day  POI  which  included  1  day  of  training  in  a  special  classroom  containing 
the  actual  tactical  equipment. 

Leibrecht  et  al.  sampled  the  traditional  instruction;  only  three  out  of  the  six  days  were  observed  for  each  of  the 
operator  classes. 
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Table  14 

Comparison  of  Instructional  Activities  by  Course  and  Topic 


Course 

Total* 

Blocks 

Lecture 

Video 

Demo 

Guided 

Demo 

Practical 

Exercise 

Review 

Testb 

Messages 

dL  1 

84 

0 

0 

3  (.04) 

41  (.49) 

14  (.17) 

9  (.11) 

17  (.20) 

dL  2C 

84 

1  (.01) 

0 

0 

35  (.42) 

20  (.24) 

28  (.33) 

0 

dL  1+2 

168 

1  (.01) 

0 

3  (.02) 

76  (.45) 

34  (.20) 

37  (.22) 

17  (.10) 

Traditional  ld 

43 

2  (.05) 

0 

0 

39  (.91) 

0 

2  (.05) 

0 

Overlays/Orders 

dL  1 

92 

0 

0 

6  (.07) 

48  (.52) 

25  (.27) 

13  (.14) 

0 

dL  2 

74 

0 

0 

0 

30  (.41) 

11  (.15) 

1  (-01) 

32  (.43) 

dL  1+2 

166 

0 

0 

6  (.04) 

78  (.47) 

36  (.22) 

14  (.08) 

32  (.19) 

Traditional  2d 

61 

5  (.08) 

0 

0 

23  (.38) 

25  (.41) 

8  (.13) 

0 

Messages  and  Overlays/Orders 

dL  1 

176 

0 

0 

9  (.05) 

89  (.51) 

39  (.22) 

22  (.13) 

17  (.10) 

dL  2 

158 

1  (.01) 

0 

0 

65  (.41) 

31  (.20) 

29  (.18) 

32  (.20) 

dL  1+2 

334 

1  (.003) 

0 

9  (.03) 

154  (.46) 

70  (.21) 

51  (.15) 

49  (.15) 

Traditional  l+2d 

104 

7  (.07) 

0 

0 

62  (.60) 

25  (.24) 

10  (.10) 

0 

“Total  blocks  =  the  number  of  blocks  that  were  dedicated  to  the  selected  topics.  Does  not  include  delays  due  to  technical  issues.  Does  not  include 
breaks. 

bTest  =  evaluation  conducted  as  part  of  the  instruction  for  that  topic  -  not  the  end-of-course  evaluation. 
cFBCB2-2  had  technical  issues  during  the  first  day  of  Paining  which  may  have  affected  the  results. 

dThese  results  are  based  on  the  observations  from  one  FBCB2  Operator  course;  the  results  for  messages  and  overlays/orders  came  from  two 
different  classes. 


As  shown  in  Table  15,  the  frequency  patterns  of  training  techniques  demonstrate 
similarities  in  the  orders/overlays  instruction  between  the  two  types  of  courses  -  although  the  dL 
instructors  also  employed  other  types  of  techniques  specific  to  the  distributed  environment  (e.g., 
taking  over  a  student’s  mouse).  However,  the  time  spent  providing  operational  examples  in  the 
dL  classes  varied  by  instructor.  Additionally,  the  dL  instructors  provided  general  computer 
examples  (general  knowledge)  and  related  new  information  to  previous  course  content  much 
more  often  than  the  traditional  instructors. 

Conducting  checks  on  student  learning  occurred  approximately  five  times  as  often  in  the 
dL  courses  compared  to  the  traditional  courses  during  the  messages  instruction.  This  suggests 
that  the  absence  of  face-to-face  feedback  led  the  dL  instructors  to  rely  more  on  learning  checks 
to  generate  feedback  on  student  progress  for  this  topic.  However,  for  the  orders  and  overlays 
topic,  the  frequencies  for  this  technique  were  similar  across  the  courses.  It  is  not  clear  why  the 
learning  check  difference  between  dL  and  traditional  courses  was  topic -specific. 

After  checking  the  raw  data,  the  large  differences  between  the  dL  and  traditional  classes 
for  giving  purpose  and  path  are  most  likely  due  to  coding  issues  and  not  instructional  differences. 

Finally,  some  techniques  were  rarely  used  in  both  types  of  classes:  encouraging  active 
learning,  pointing  to  screen  prompts,  emphasizing  practice,  and  using  memory  aids.  Although 
both  dL  instructors  invited  the  students  to  arrive  early  or  remain  after  the  scheduled  end  of  class 
so  they  could  practice  and  ask  questions,  only  one  student  stayed  to  work  with  the  instructor  for 
about  45  minutes.  No  other  cases  of  extra  student  practice  were  recorded.  The  students  did  not 
create  their  own  practice  sessions  by  arriving  early  or  working  during  breaks,  as  happened  in  the 
research  reported  by  Leibrecht  et  al.  (2007). 
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Table  15 

Comparison  of  Training  Techniques  by  Course  and  Topic 


Training  Techniques 

Course 

Point  to 
Screen 
Prompts 

Emphasize 

Practice 

Use 

Memory 

Aids 

Check 

Learning 

Progress 

Give 
Purpose 
&  Path 

Relate  to 
Military 
Ops 

Relate  to 
General 
Knowledge 

Relate  to 
Previous 
Content 

Respond 

to 

Learners 

Encourage 

Active 

Learning 

Use  dL 
Techniques 

Messages 

dL  1 

2  (.02) 

1  (-01) 

0 

38  (.45) 

0 

11  (.13) 

39  (.46) 

5  (.06) 

3  (.04) 

0 

49  (.58) 

dL  2 

0 

3  (.04) 

0 

46  (.55) 

0 

18  (.21) 

35  (.42) 

1  (-01) 

5  (.06) 

0 

36  (.43) 

dL  1+2 

2  (.01) 

4  (.02) 

0 

84  (.50) 

0 

29  (.17) 

74  (.44) 

6  (.04) 

8  (.05) 

0 

85  (.51) 

Trad  1 

0 

0 

0 

4  (.09) 

38  (.88) 

12  (.28) 

0 

0 

0 

0 

NA 

Orders  and  Overlays 

dL  1 

0 

0 

0 

27  (.29) 

0 

21  (.23) 

44  (.48) 

5  (.05) 

19  (.21) 

9  (.10) 

55  (.60) 

dL  2 

0 

0 

0 

25  (.34) 

0 

0 

30  (.41) 

18  (.24) 

13  (.18) 

0 

31  (.42) 

dL  1+2 

0 

0 

0 

52  (.31) 

0 

21  (.13) 

74  (.45) 

23  (.14) 

32  (.19) 

9  (.05) 

86  (.52) 

Trad  2 

0 

0 

1  (.02) 

18  (.30) 

19  (.31) 

11  (.18) 

1  (.02) 

5  (.08) 

9  (.15) 

0 

NA 

Messages,  Orders  and  Overlays 

dL  1 

0 

1  (.01) 

0 

65  (.37) 

0 

32  (.18) 

83  (.47) 

10  (.06) 

22  (.13) 

9  (.05) 

104  (.59) 

dL  2 

0 

3  (.02) 

0 

71  (.45) 

0 

18  (.11) 

65  (.41) 

19  (.12) 

18  (.11) 

0 

67  (.42) 

dL  1+2 

0 

4  (.01) 

0 

136  (.41) 

0 

50  (.15) 

148  (.44) 

29  (.09) 

40  (.12) 

9  (.03) 

171  (.51) 

Trad 

1+2 

0 

0 

1  (.01) 

22  (.21) 

57  (.55) 

23  (.22) 

1  (.01) 

5  (.05) 

9  (.09) 

0 

NA 

Student  Behaviors 


The  set  of  fifteen  student  behaviors  ranged  from  simple  actions — such  as  asking  a 
question  and  waiting  for  others —  to  working  in  groups  and  coaching  others  (see  the  Method 
section  for  a  complete  description).  These  behaviors  formed  the  categories  used  to  classify 
student  actions  in  the  classroom.  This  section  describes  the  student  behaviors.  It  also  presents 
the  student  opinion  data  obtained  by  administering  the  feedback  questionnaire. 

The  methodology  called  for  an  observer  to  record  every  behavior  that  occurred  at  least 
once  in  a  5-min  block,  using  pre-defined  codes.  In  the  majority  of  cases  the  observer  recorded 
two  or  even  three  codes  in  a  5-min  block. 

Table  16  summarizes  the  frequency  data  for  the  various  student  behaviors.  The  most 
frequent  behaviors  overall  were  working  alone  (52%  of  the  blocks)  and  paying  attention  to  the 
instructor  (50%  of  the  blocks).  Next  most  frequent  was  coaching  peers  (40%  of  the  blocks), 
followed  by  asking  questions  (21%  of  the  blocks).  For  the  rest  of  the  behaviors  the  frequencies 
fell  substantially,  to  once  in  17  blocks  or  less. 

With  three  exceptions,  the  relative  frequencies  of  student  behaviors  were  very  similar  for 
the  two  courses  (separated  by  10  percentage  points  or  less).  In  FBCB2-1,  paying  attention  to  the 
instructor  occurred  more  frequently  (in  57%  of  the  blocks)  than  in  FBCB2-2  (42%  of  the  blocks). 
Conversely,  two  behaviors  occurred  more  frequently  in  FBCB2-2 — asking  questions  (33%  vs. 
10%  of  the  blocks,  respectively)  and  working  alone  (61%  vs.  45%  of  the  blocks).  The  reasons 
for  these  differences  are  unclear,  but  various  student  factors  (and  perhaps  student-instructor 
interactions)  probably  played  a  role.  Variations  in  student  behaviors  are  to  be  expected. 

Close  inspection  of  Table  16  reveals  a  few  trends  across  days  (excluding  the  exam- 
focused  Day  5).  Paying  attention  to  the  instructor  declined  modestly  across  days  in  both  courses. 
On  the  other  hand,  working  alone  increased  noticeably  across  days  in  both  courses,  as  did 
coaching  peers.  These  trends  may  be  the  result  of  student  confidence  increasing  as  the  course 
progressed.  Further,  asking  questions  increased  across  days  in  FBCB2-2  but  not  FBCB2-1.  For 
both  classes,  responding  to  questions  occurred  much  less  frequently  than  asking  questions. 

The  summed  data  in  the  last  row  of  Table  16  provide  a  basis  for  charting  general  trends 
in  the  student  behaviors.  In  descending  order  from  most  frequent  to  least,  the  following  rank 
order  of  student  behaviors  emerged:  working  alone,  paying  attention  to  the  instructor,  coaching 
peers,  asking  questions,  waiting  for  other  students,  engaging  in  side  conversations,  working  in 
groups,  and  responding  to  questions.  The  latter  four  behaviors  were  observed  infrequently  (in  4- 
6%  of  the  blocks).  The  rest  of  the  behaviors  occurred  rarely  (in  1%  of  the  blocks  or  less). 
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Table  16 

Frequency  of  Observed  Student  Behaviors 


Day 

Pay  Attn 
to 

Instructor 

Ignore 

Instructor 

Ask 

Question 

Respond 

to 

Question 

Ask  to 
Repeat 
Step 

Release 

Mouse 

Control 

Lag 

Behind 

Instructor 

Work 
Ahead  of 
Instructor 

Work 

Alone 

Work  in 
Groups 

Converse 

Aside 

Coach 

Peers 

Use 

Other 

Functions 

Wait  for 
Others 

Other 

Behaviors 

FBCB2  Distributed  Course  #1 

1 

49  (.71) 

0 

6  (.09) 

11  (.16) 

0 

1  (.01) 

1  (.01) 

0 

16  (.23) 

3  (.04) 

1  (.01) 

13  (.19) 

1  (.01) 

4  (.06) 

1  (.01) 

2 

49  (.74) 

1  (.02) 

12  (.18) 

0 

1  (.02) 

0 

1  (.02) 

0 

23  (.35) 

0 

4  (.06) 

25  (.39) 

0 

0 

3  (.05) 

3 

43  (.62) 

0 

4  (.06) 

0 

0 

3  (.04) 

1  (.01) 

4  (.06) 

26  (.38) 

2  (.03) 

6  (.09) 

43  (.62) 

4  (.06) 

1  (-01) 

1  (.01) 

4 

40  (.59) 

0 

9  (.13) 

0 

0 

3  (.04) 

1  (.01) 

1  (-01) 

29  (.43) 

8  (.12) 

0 

26  (.38) 

0 

0 

1  (.01) 

5 

11  (.16) 

3  (.04) 

4  (.06) 

0 

0 

0 

0 

0 

57  (.85) 

0 

0 

21  (.31) 

0 

0 

0 

1-5 

192  (.57) 

4  (.01) 

35  (.10) 

11  (.03) 

1  (.003) 

7  (.02) 

4  (.01) 

5  (.01) 

151  (.45) 

13  (.04) 

11  (.03) 

128  (.38) 

5  (.01) 

5  (.01) 

6  (.02) 

FBCB2  Distributed  Course  #2 

1 

30  (.86) 

0 

9  (.26) 

0 

0 

1  (.03) 

2  (.06) 

1  (.03) 

5  (.14) 

13  (.37) 

6  (.17) 

6  (.17) 

0 

2  (.06) 

0 

2 

34  (.45) 

0 

23  (.30) 

0 

0 

2  (.03) 

0 

0 

43  (.55) 

0 

9  (.12) 

38  (.50) 

0 

16  (.21) 

0 

3 

33  (.42) 

0 

24  (.30) 

1  (.01) 

0 

2  (.03) 

0 

0 

53  (.67) 

0 

0 

68  (.86) 

0 

8  (.10) 

1  (.01) 

4 

31  (.54) 

0 

37  (.65) 

3  (.05) 

0 

2  (.03) 

1  (.02) 

0 

28  (.49) 

0 

4  (.07) 

18  (.32) 

0 

8  (.14) 

0 

5 

0 

0 

9  (.16) 

10  (.17) 

0 

0 

0 

0 

58  (1.0) 

0 

2  (.03) 

0 

0 

0 

0 

1-5 

128  (.42) 

0 

102  (.33) 

14  (.05) 

0 

7  (.02) 

3  (.01) 

1  (.003) 

187  (.61) 

13  (.04) 

21  (.07) 

130  (.43) 

0 

34  (.11) 

1  (.003) 

All  Courses/Days  (Summed) 

All 

320  (.50) 

4  (.006) 

137  (.21) 

25  (.04) 

1  (.002) 

14  (.02) 

7  (.01) 

6  (.009) 

338  (.52) 

26  (.04) 

32  (.05) 

258  (.40) 

5  (.008) 

39  (.06) 

7  (.01) 

Student  Survey  Results 

This  subsection  summarizes  the  data  obtained  in  the  survey  questionnaires  completed  by 
students  (Appendix  D)  at  the  end  of  each  course.  Data  tables  are  presented  in  Appendix  F. 

For  both  dL  classes,  the  results  of  the  student  survey  indicated  similar  responses 
regarding  the  following  aspects  of  the  course  (see  Appendix  F,  Table  FI  for  complete  results): 

•  Pace  of  instruction  was  appropriate  (50%  each), 

•  Did  not  work  ahead  of  instruction  or  only  during  certain  topics  (-80%  each), 

•  Sought  assistance  from/provided  assistance  to  fellow  students  (83-100%), 

•  Instructors  answered  all  of  their  questions  (83-90%),  and 

•  Engaged  and  motivated  to  learn  throughout  the  course  (68-80%). 

However,  the  students  differed  across  the  two  dL  classes  regarding  the  extent  to  which 
they  felt  that  they  learned  the  system  (40%  vs.  83.3%)  and  the  extent  to  which  they  felt  that  the 
dL  format  facilitated  learning  (10%  vs.  50%;  see  Appendix  L,  Table  LI).  The  following  are 
student  comments  for  how  the  dL  format  helped  learning: 

•  Better  than  a  regular  classroom  environment  because  the  instructor  can  monitor  all  of  the 

computer  screens  and  directly  access  student  data. 

•  The  instructor  was  able  to  answer  all  questions  effectively. 

On  the  other  hand,  the  students  noted  that  technical  difficulties  and  the  lack  of  L2L 
interaction  were  some  of  the  factors  that  hindered  learning  in  the  dL  classroom  environments. 
The  following  are  student  comments  regarding  these  issues: 

•  Technical  difficulties. 

-  Lag  time  /  system  malfunctions  slowed  learning. 

-  Noise  interference;  difficult  to  hear  instructions  at  times. 

-  Computer  problems  made  it  difficult  to  keep  pace  with  instructor. 

-  Many  people  talking  at  the  same  time;  no  way  to  see  who  is  speaking. 

-  Screen  image  very  blurry  on  instructor  end. 

•  Lack  of  L2L  interaction  slowed  progress  when  technical  problems  occurred. 

Additionally,  the  students  were  asked  to  indicate  whether  specific  LBCB2  procedures 
were  easy  or  difficult  to  learn  (see  Appendix  L,  Table  L2).  Preparation  for  combat  procedures 
and  creating/sending  messages  were  easy  to  learn  for  one  class;  the  other  class  was  neutral.  The 
students  indicated  that  it  was  easy  to  learn  LBCB2  functions  because  of  the  instructional  methods 
(e.g.,  good  instructor,  repetition),  the  system  design  (e.g.,  information  was  well  organized  and 
the  system  was  easy  to  use),  and  prior  experience  using  ABCS  systems. 

The  results  indicated  that  developing/sending  orders  and  overlays  tended  to  be  more 
difficult  to  learn  for  one  class;  the  other  class  was  relatively  neutral  (see  Appendix  L,  Table  L2). 
Some  students  indicated  that  it  was  difficult  to  leam  specific  LBCB2  functions  because  of  a  lack 
of  prerequisite  knowledge,  especially  computer  experience  or  familiarity  with  operational  or 
battlefield  terminology.  The  comments  also  suggested  that  technical  terms  were  used  instead  of 
common  terms  and  recommended  that  the  instructors  relate  the  technical  terms  to  the  course. 
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The  students  also  felt  that  additional  training  aids  would  have  increased  overall 
leaming/understanding,  such  as  a  student  manual  with  complete  instructions  for  beginners  and 
access  to  real  systems  with  some  vehicle  time  as  part  of  the  training.  Finally,  the  students 
commented  that  the  amount  of  information,  especially  in  regards  to  menus,  submenus,  and 
overlays,  posed  challenges  to  learning.  However,  some  students  felt  that,  overall,  with  all  the 
reinforcement  tools,  it  was  a  great  course. 

The  survey  results  also  indicated  that  the  majority  of  the  students  felt  that  demonstration 
with  student  participation ,  practical  exercises ,  review  of  material,  and  testing  were  effective 
techniques.  A  few  of  the  recommended  instructional  approaches  for  the  dL  environment 
included  practice  with  an  actual  FBCB2  system  on  site,  more  challenging  PEs,  and  additional 
quizzes.  On  the  other  hand,  some  individuals  felt  that  the  instructor  successfully  guided  them 
through  all  of  the  instruction. 

General  comments  concerning  what  the  students  liked  most  about  the  course  included  the 
following: 

•  Convenience  -  not  having  to  travel. 

•  Unit  organization  -  being  able  to  stand  up  a  class  just  for  our  unit. 

•  Peers  -  work  with  others;  help  from  classmates. 

•  Instructional  approaches  /  material  -  well  organized;  good  pace,  the  material  being 
taught,  the  hands-on  instruction,  and  the  ability  of  instructor  to  walk  me  though 
something  on  my  own  computer. 

•  That  we  were  able  to  get  the  instruction  at  all. 

•  Enjoyed  the  course  /  Very  good  class  and  instructor. 

Recommendations  for  course  improvements  included  the  following: 

•  Fix  Technical  Difficulties. 

-  Connectivity  /  Upgrade  internet  service. 

-  System  slowdowns  and  lock  ups.  Lag  in  the  systems.  Time  delays. 

-  Communication  issues  -  audio/visual  problems. 

-  Improve  the  computer  software. 

•  Hands  On  Experience. 

-  Really  need  hands  on  -  how  do  you  turn  this  thing  on? 

-  Have  the  real  gear  onsite.  Having  the  system  here  itself  to  observe. 

-  Hands  on  field  training. 

•  Better  Training  Aids. 

-  Better  course  book/workup  that  doubles  as  a  commander’s  guidebook  and  is 
useful  to  train  our  Soldiers;  better  train  the  trainer  focus. 

-  More  training  aids. 

•  No  changes  -  /  wouldn ’t  change  a  thing. 

Finally,  the  students  reported  mixed  results  when  comparing  the  dL  LBCB2  class  to 
traditional  ABCS  classes.  That  is,  some  students  felt  that  there  were  no  differences  between  the 
two  types  of  classes  while  other  students  felt  that  it  was  more  effective  or  would  be  more 
effective  than  the  traditional  training  if  the  communication  issues  were  resolved.  On  the  other 
hand,  some  students  felt  that  the  dL  classes  were  less  effective  than  the  traditional  training  and 
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needed  additional  training  aids.  In  particular,  one  student  commented  that  although  the 
instructor  can  take  control  of  the  students’  systems  when  they  need  help,  it  is  not  as  effective  as 
letting  the  students  do  it  on  their  own  with  the  instructors  guiding.  Finally,  another  student 
emphasized  the  need  for  training  on  FBCB2  components  in  a  tactical  military  vehicle  and  that 
residential  training  provides  this  type  of  training. 

Additional  Instructor  Responses  Regarding  dL  Classroom  Environments 

During  one-on-one  interviews,  the  instructors  indicated  that  the  virtual  classroom  as 
equipped  is  effective  in  supporting  student  learning.  However,  they  felt  that  the  most  limiting 
factors  of  the  dL  environment  were  the  insufficient  bandwidth  and  the  unreliability  of  Internet 
connectivity.  Both  factors  result  in  significant  distractions  for  the  instructor  and  students.  The 
insufficient  bandwidth  typically  produces  a  lag  in  FBCB2  interaction  during  the  normal  conduct 
of  classes,  while  the  unreliable  Internet  connectivity  produces  unplanned  interruptions  of  varying 
duration.  When  asked  what  could  be  done  to  improve  the  capabilities,  both  instructors  stated 
that  greater  bandwidth  and  highly  reliable  connectivity  were  essential. 

One  instructor  stated  that,  if  the  bandwidth  and  Internet  reliability  issues  were  corrected, 
dL  classes  “definitely  would  be  the  way  to  go.”  Both  instructors  cited  advantages  to  using  the 
dL  method  (see  Table  17).  These  advantages  may  promote  effective  instruction,  at  least  in  the 
initial  stages  of  a  course. 

Table  17 

Instructor  View  of  Distributed  Learning  Advantages 


Advantage 

Explanation 

Improved  monitoring 
of  student  learning 

With  the  ability  to  observe  each  student’s  screen,  the  instructor  can 
determine  when  to  give  individual  attention  or  adjust  the  pace  to 
accommodate  student  progress. 

Reduced  overhead 

The  elimination  of  instructor  travel  yields  significant  savings  of 
time  and  costs.  Similarly,  the  instructor  is  not  required  to  spend 
time  setting  up  a  classroom  and  preparing  for  a  new  setting. 

Instructor  “comfort 
zone” 

The  instructor  is  able  to  teach  in  a  familiar  environment  without 
adjusting  to  a  new  setting.  Also,  the  instructor  can  resort  to  notes 
without  the  students  being  aware  he  is  doing  so. 

Multimedia  integration 

The  VTT  equipment  provides  a  single  platform  for  all  multimedia 
materials,  eliminating  the  need  for  other  equipment  such  as  an 
electronic  projector. 

Greater  student 
capacity 

Because  classes  are  not  limited  to  a  specific  location,  more 
students  can  be  taught  in  different  classes  across  the  US  or  outside 
of  the  US.  (The  multiple  Instructor  Booths  can  provide  instruction 
simultaneously  with  less  logistical  issues  than  having  instructors 
travel  to  multiple  locations). 
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The  instructors  also  discussed  some  of  the  difficulties  associated  with  the  lack  of  F2F 
interaction.  For  example,  assisting  students  who  need  additional  help,  especially  those  without 
sufficient  digital  skills,  can  be  a  challenge.  Although  this  occurs  in  traditional  courses  as  well, 
the  instructor  stated  that  the  situation  was  more  difficult  to  overcome  since  he  could  not  provide 
F2F  assistance  to  the  students.  In  the  absence  of  an  AI  in  the  classroom,  as  occurred  in  both 
courses  observed,  students  with  better  computer  skills  assisted  other  students.  A  related  issue  is 
the  difficulty  of  accomplishing  robust  instructor-student  interaction,  as  compared  to  the 
residential  classroom,  which  helps  to  keep  students  focused. 

To  improve  the  course,  the  instructors  indicated  that  the  class  could  be  shortened  by 
cutting  out  less  important  topics  or  by  breaking  the  class  down  into  phases  to  reduce  the  timeline 
for  training.  Other  ideas  mentioned  were  to  tailor  the  class  to  the  students’  basic  needs  and/or 
teach  only  core  system  functions.  One  instructor  also  suggested  introducing  more  plug  and  play 
sustainment  or  refresher  training  where  the  students  can  log  into  the  system  on  their  own. 
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DISCUSSION 


The  current  research  investigated  the  blended  learning  approaches  used  to  train  digital 
skills  in  dL  classroom  environments.  The  specific  goals  were  to  1)  report  lessons  learned  from 
instructors  in  a  distributed  blended  learning  environment;  2)  report  student  reactions  to  this  type 
of  learning  environment;  and  3)  compare  the  training  approaches  with  those  reported  for 
traditional  classroom  environments. 

Effective  Techniques  for  clL  Classroom  Environments 

The  results  of  this  research  extend  the  knowledge  base  on  instructional  practices  and 
training  techniques  used  in  courses  that  build  digital  operator  skills.  The  dL  instructors  applied  a 
variety  of  techniques  to  convey  course  content,  maintain  the  students’  motivation,  and  respond  to 
students’  problems  and  concerns.  In  particular,  the  following  techniques  were  especially 
effective  for  the  dL  environment. 

First,  the  instructors  used  particular  students,  who  had  mastered  certain  procedures,  to 
demonstrate  the  procedures  to  other  students.  This  proved  to  be  an  effective  technique  in  the 
absence  of  an  on-site  AI  and  was  valuable  when  the  bandwidth  was  limited.  If  possible,  the 
instructors  could  identify  these  individuals  prior  to  the  start  of  the  class,  so  that  immediate 
assistance  could  be  provided.  Ideally,  an  AI  or  experienced  FBCB2  operator  from  the  students’ 
unit  could  attend  the  class  and  offer  assistance  and  share  experiences  of  how  they  used  certain 
features  while  deployed.  This  would  be  of  great  value  to  the  dL  instructors  as  the  lack  of  F2F 
interaction  makes  it  difficult  for  the  instructors  to  keep  the  students  engaged,  motivated,  and 
focused,  especially  when  there  are  technical  problems. 

Second,  we  observed  peer  coaching  during  many  of  the  course  topics.  The  instructors 
mentioned  that  they  encourage  peer  coaching.  To  structure  these  interactions  so  that  they  do  not 
interfere  with  learning,  the  instructors  could  ask  the  students  to  work  on  a  problem  for  a 
specified  period  of  time  before  asking  their  fellow  classmates  for  help.  After  receiving 
assistance,  the  students  should  be  given  the  opportunity  to  practice  the  procedures  on  their  own 
so  that  they  will  be  better  able  to  retain  the  information. 

Third,  the  instructors  employed  a  few  techniques  to  more  effectively  monitor  the  students’ 
performance.  A  bank  of  monitors  showed  each  student’s  computer  screen,  and  allowed  the 
instructors  to  view  the  progress  of  each  student  as  they  worked  through  the  FBCB2  procedures. 
They  also  used  computer  software  that  allowed  them  to  take  over  the  mouse  of  a  student  who 
was  having  difficulty  with  the  procedures.  This  way,  the  student  received  individualized 
instruction  as  the  instructor  walked  through  the  correct  steps  of  the  procedure  from  where  the 
student  had  left  off.  However,  we  note  that  this  technique  had  differential  effects  on  student 
learning.  Some  students  felt  that  this  technique  helped  them  to  learn  the  difficult  procedures 
while  other  students  felt  that  this  approach  was  not  as  effective  as  when  they  were  encouraged  to 
perform  the  steps  on  their  own  with  instructor  guidance.  We  provide  some  recommendations 
below  regarding  these  issues. 
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Comparison  to  Traditional  Classes 


Overall,  the  findings  indicate  that  the  dL  instructors  were  able  to  adapt  to  using  the 
technology  to  teach  FBCB2  operations,  resulting  in  training  that  is  very  comparable  to  the 
traditional  courses.  The  positive  student  comments  and  similar  training  approaches  and  topical 
coverage  support  these  findings.  Thus,  training  developers  and  instructors  in  need  of  alternative 
ways  to  provide  training  on  ABCS  components  could  successfully  employ  the  dL  method. 

Although  the  overall  patterns  of  dL  activities  and  techniques  were  similar  to  those  found 
in  traditional  classrooms,  specific  adaptations  occurred  to  accommodate  the  dL  environment. 
Overall,  the  dL  instructors  spent  much  more  time  covering  the  topics  of  messages  and 
orders/overlays  than  the  instructors  in  the  traditional  classes.  The  absence  of  F2F  interaction 
may  have  led  the  instructors  to  conduct  more  learning  checks  to  generate  feedback  on  student 
progress,  which  may  account  for  some  of  the  additional  time.  The  checks  on  learning,  review  of 
material,  and  testing  are  all  effective  ways  to  ensure  that  students  understand  the  material  and  are 
learning  the  skills,  and  they  should  be  sustained. 

The  dL  instructors  also  provided  many  more  general  computer  examples  and  related  the 
information  to  previous  course  content  much  more  often  than  the  residential  instructors. 
Providing  general  computer  examples  definitely  makes  it  easier  for  the  students  to  understand 
the  functions.  On  the  other  hand,  the  dL  instructors  differed  in  the  extent  to  which  they  used 
operational  examples  throughout  the  course;  a  recommendation  for  utilizing  this  technique  more 
when  teaching  digital  skills  is  discussed  in  more  detail  below. 

It  is  important  to  note  that  although  some  differences  were  found  between  the  dL  and 
traditional  classes,  these  may  have  been  due  to  other  factors  besides  the  training  medium  (e.g., 
background  of  students  in  the  class,  etc.).  Specifically,  although  the  POIs  for  the  dL  and 
traditional  courses  are  basically  the  same,  the  instructors  may  modify/adopt  a  different  emphasis 
based  on  student  differences.  Thus,  the  differences  we  found  between  the  two  courses  may  be 
due  to  the  way  in  which  the  instructors  adapted  to  the  needs  of  a  particular  class.  Additionally,  as 
only  one  traditional  class  was  observed  per  topic,  the  number  of  observations  was  lower  for  the 
traditional  courses  compared  to  the  dL  courses  in  which  two  classes  were  observed  per  topic. 
These  differences  in  the  frequencies  of  the  observations  could  have  affected  the  results.  In 
summary,  we  present  the  comparison  findings  as  a  case  study  of  the  possible  differences  across 
these  different  training  environments  and  note  that  additional  research  is  needed  that  controls  for 
some  of  these  factors  to  draw  strong  conclusions  from  the  results. 


Major  Advantages  and  Disadvantages  of  the  dL  Approach 

The  dL  instructors  reported  noteworthy  advantages  to  providing  FBCB2  operator  training 
using  the  distributed  method.  In  particular,  they  felt  that  the  reduced  cost,  especially  in  terms  of 
travel,  and  the  ability  to  deliver  the  training  to  multiple  classes  simultaneously  throughout  the 
nation  is  a  large  benefit  of  the  approach.  The  student  survey  results  supported  these  comments 
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as  some  students  appreciated  the  opportunity  to  receive  the  remote  training  because  they  may  not 
have  received  the  training  otherwise. 

Although  the  findings  suggest  that  the  instructors  were  able  to  overcome  the 
technological  difficulties,  one  disadvantage  of  the  dL  approach  concerns  the  issues  with  the 
insufficient  bandwidth  and  the  unreliability  of  Internet  connectivity.  Both  factors  resulted  in 
significant  distractions  for  the  instructor  and  students.  To  effectively  use  the  class  time  when 
these  technological  issues  occur,  the  instructors  could  have  alternative  exercises  or  course 
material  available.  For  example,  although  not  ideal,  the  instructors  could  walk  the  students 
through  a  demonstration  of  the  FBCB2  steps  via  screen  shots  embedded  in  PowerPoint  slides. 
Most  screen  shots  are  already  provided  to  the  students  at  the  start  of  the  class  as  part  of  the 
reference  packet.  The  instructors  also  could  conduct  PEs  either  through  verbal  discussions  or 
paper-pencil  materials,  focusing  on  when  certain  procedures  would  be  used  while  deployed. 

Further,  research  suggests  that  individuals  who  provide  summaries  of  what  they  have 
learned  show  10%  gains  over  those  individuals  who  did  not  summarize  (for  a  review  see  Abell, 
2000).  Instructors  could  use  the  down-time  to  ask  students  to  summarize  certain  procedures.  If 
actual  systems  were  available,  this  would  be  a  good  time  for  the  students  to  familiarize 
themselves  with  the  equipment.  Perhaps  videos  could  be  used  that  describe  the  actual  equipment 
or  provide  instances  of  how  different  FBCB2  features  were  used  on  deployments.  In  summary, 
the  lack  of  F2F  interaction  poses  additional  challenges  for  the  instructors  when  technological 
problems  result  in  long  lags  and  down-time,  and  they  may  be  better  able  to  keep  the  students 
engaged  by  having  alternative  materials  available. 

Student  Results 

The  survey  results  indicated  large  differences  across  the  classes  reflecting  the  extent  to 
which  the  students  felt  that  they  learned  the  system  and  the  extent  to  which  the  dL  format 
affected  their  learning.  These  differences  seem  to  be  the  result  of  one  class  experiencing 
significantly  more  technical  problems  than  the  other  class  such  as  audio/visual  problems, 
connectivity  issues,  lag  time,  noise  interference,  etc.  These  results  highlight  our  note  above  that 
one  disadvantage  to  the  dL  method  is  the  effect  of  the  technological  issues  on  learning. 

However,  in  the  present  research,  the  trends  for  working  alone,  helping  peers,  and  paying  less 
attention  to  instructors  may  reflect  increasing  student  confidence  as  the  course  progressed. 

Another  trend  we  noted  for  both  classes  was  that  responding  to  questions  asked  by  the 
instructor  occurred  much  less  frequently  than  asking  questions.  That  is,  the  students  asked  many 
more  questions  than  they  answered.  Some  research  suggests  that  questioning  by  the  instructor  is 
the  single  most  influential  teaching  behavior  that  affects  learning  and  is  key  to  fostering  student- 
instructor  interactions  (Abell,  2000).  Thus,  dL  instructors  could  actively  engage  the  students 
throughout  the  course  by  asking  them  specific  questions  about  certain  steps  of  LBCB2 
procedures  and  how  they  would  apply  the  procedures.  As  indicated  previously,  the  students  may 
better  retain  the  information  by  summarizing  the  procedures  in  their  own  words.  These 
techniques  could  be  especially  useful  for  increasing  student-teacher  interactions  in  the  dL 
environment,  especially  when  the  system  is  experiencing  technical  problems. 
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Additionally,  some  students  reported  having  difficulty  understanding  the  acronyms  used 
in  the  class.  The  observer  indicated  that  he  did  not  think  that  any  one  area  of  terminology/ 
acronyms  was  more  problematic.  The  survey  responses  regarding  difficulty  with  acronyms  came 
from  the  FBCB2-1  class,  and  our  survey  results  indicated  that  this  class  consisted  of  junior 
officers  and  NCOs  and  that  only  50%  were  very  comfortable  using  a  computer.  Thus,  the 
students  may  have  had  difficulty  with  both  operational  and  general  computer  terminology.  The 
need  for  additional  training  aids  such  as  remedial  materials  for  beginners  is  discussed  in  more 
detail  below. 

The  Soldiers  did  not  create  their  own  practice  sessions  by  arriving  early  or  working 
during  breaks,  as  happened  in  the  traditional  classes.  This  occurred  even  though  the  dL 
instructors  offered  these  options  to  the  students.  Thus,  the  instructors  may  need  to  make  time 
available  in  class  for  the  students  to  practice  complex  procedures.  Some  of  the  FBCB2  tasks  are 
difficult  to  learn  and  remember;  additional  practice  opportunities  would  help  the  students  to 
better  retain  the  information  and  transfer  the  knowledge  and  skills  to  other  situations  while 
deployed.  This  is  discussed  in  more  detail  below. 

Some  students  reported  that  they  were  attending  the  class  to  be  able  to  train  other 
Soldiers  in  their  units.  As  such,  they  indicated  that  they  would  like  additional  train-the-trainer 
materials  or  a  commander’s  guide  so  that  they  can  better  teach  the  FBCB2  procedures.  The  dL 
instructors  could  provide  them  with  some  of  the  exercises  that  they  use  in  the  class  so  that  other 
Soldiers  have  the  opportunity  to  practice  the  procedures. 

Finally,  one  recommendation  made  repeatedly  by  the  students  was  for  the  class  to  spend 
some  time  training  with  the  real  FBCB2  systems  (“greenboxes”).  The  students  wanted  some 
hands-on  experience  with  real  systems  so  they  could  familiarize  themselves  with  the  equipment 
(e.g.,  turning  it  on  and  off).  By  incorporating  training  with  the  real  equipment  into  the  course, 
the  dL  instructors  would  likely  increase  the  transfer  of  the  digital  skills  learned  in  the  course  to 
operational  settings  (cf.  Laker,  1990).  However,  at  the  very  least,  it  would  require  the  remote 
classrooms  to  have  greenboxes  set-up  and  in  working  condition  for  the  students  to  use. 

Providing  this  could  significantly  increase  the  cost  of  the  dL  course. 

Lessons  Learned  /  Training  Recommendations 

Although  the  dL  instructors  incorporated  some  of  the  following  techniques  in  the  dL 
method  of  FBCB2  instruction,  we  offer  additional  ways  in  which  the  instruction  could  be 
modified  to  enhance  learning  and  transfer  of  the  digital  skills  to  operational  contexts. 

The  recommendations  contained  in  this  section  of  the  report  reflect  findings  from  the 
literature  review  conducted  for  the  current  project  (as  described  in  the  Introduction),  the  current 
project’s  results,  and  training  recommendations  from  a  forthcoming  ARI  report  and  compact  disk 
(cd)-based  tool  for  digital  skills  instructors  (Blankenbeckler,  Wampler,  Goodwin,  &  Dyer,  in 
preparation).  These  recommendations  also  parallel  the  findings  from  Abell  (2000)  regarding  the 
needs  of  Soldiers  as  distance  learners  and  those  of  Sanders  (2000)  regarding  constructivist 
principles  and  related  training  techniques. 
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First,  we  suggest  that  there  is  not  an  optimal  mix  of  training  techniques  -  it  depends  on 
the  level  of  skill  of  the  students  which  varies  from  class  to  class.  A  good  approach  is  to  train 
instructors  to  be  knowledgeable  about  the  different  training  techniques  and  when  to  use  them. 
With  this  in  mind,  we  offer  a  few  specific  recommendations  for  dL  instructors  reflecting  the 
following  eight  areas: 

1.  Provide  materials/instruction  for  student  prerequisites, 

2.  Leverage  student  strengths  and  experience, 

3.  Train  functions  in  a  system- wide/realistic  context, 

4.  Use  PEs  to  foster  problem-centered  learning, 

5.  Leverage  observe-only  demonstration, 

6.  Expand  practice  opportunities, 

7.  Emphasize  independent  practice,  and 

8.  Provide  ample  training  aids. 

1.  Provide  materials/instruction  for  student  prerequisites.  Enrollment  in  the  dL  operator 
courses  did  not  depend  on  any  apparent  prerequisites.  However,  a  minimum  level  of  digital 
and  tactical  competency  is  essential  for  students  to  optimize  their  learning  and  avoid  causing 
class  interruptions  or  delays. 

a.  Importance  of  enablers.  Because  FBCB2  is  a  computer  system,  some  general 
familiarity  or  comfort  level  with  computers  is  imperative  for  the  students.  Less 
obvious  is  knowledge  of  tactics,  techniques,  and  procedures  (TTP),  tactical/technical 
terminology  and  associated  acronyms.  Basic  tactical  knowledge  is  essential  to 
FBCB2  operation  and  to  understanding  the  system’s  use  in  the  field.  For  example,  a 
student  who  is  not  familiar  with  military  overlays  that  include  boundaries  and  phase 
lines  will  find  it  difficult  to  leam  FBCB2’s  overlay  features.  If  a  student  does  not 
understand  foundation  definitions,  concepts  and  implications,  then  the  instruction 
occurs  in  the  abstract  and  learning  to  use  the  FBCB2  has  no  frame  of  reference. 

b.  Default  assumptions .  Given  the  students’  diversity  in  terms  of  rank,  experience,  and 
branch/military  occupational  specialty,  the  instructor  should  assume  that  some 
students  lack  the  requisite  skills  and  knowledge,  especially  acronyms  and 
terminology  related  to  tactical  operations  and  computer  interfaces.  At  the  start  of  a 
course,  a  good  step  would  be  for  the  instructor  to  assess  the  students’  knowledge  of 
terminology,  acronyms,  computers,  and  military  tactics. 

c.  Pre-training.  A  means  of  imparting  the  requisites  in  advance  would  improve  learning 
during  the  course.  In  the  case  of  TTP,  terminology  and  acronyms  this  could  be  a 
simple  read-ahead  packet  provided  well  before  the  course  and  reinforced  by  a  review 
on  the  first  day.  In  the  case  of  computer  skills,  the  instructor  must  quickly  identify 
students  who  require  additional  training  and  provide  it  at  the  first  opportunity.  For 
example,  he  could  hold  a  computer  skills  class  during  lunch  or  after  class  on  the  first 
day  of  the  course. 

2.  Leverage  student  strengths  and  experience.  Determine  start-of-course  qualifications.  At  the 
beginning  of  the  class,  the  instructors  could  find  out  the  backgrounds  and  experience  of  the 
students  and  build  on  what  the  students  already  know. 
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a.  For  example,  if  the  students  have  Infantry  backgrounds,  the  instructors  could  relate 
some  of  the  FBCB2  functions  to  land  navigation  -  the  students  are  used  to  navigating 
on  the  ground  with  GPS,  now  they  have  to  learn  how  to  navigate  using  FBCB2. 

b.  Or,  if  there  are  differences  between  an  alternate  communication  method  and  FBCB2, 
then  point  out  the  major  discrepancies  to  reduce  negative  transfer.  For  example, 
discuss  the  big  differences  between  9-Line  Medical  Evacuation  (MEDEVAC) 

Request  and  a  MEDEVAC  Request  using  FBCB2. 

c.  Instructors  can  capitalize  on  the  students’  experiences  by  encouraging  them  to  share 
real-world,  relevant  examples  (Abell,  2000). 

3.  Train  functions  in  a  system-wide/realistic  context.  As  much  as  possible,  digital  functions 
should  be  trained  in  context;  they  should  not  be  viewed  as  independent  of  each  other.  They 
should  be  integrated  in  sequence,  and  students  should  practice  them  in  reaction  to  an  event 
that  would  occur  in  operational  use.  Thus,  the  training  tasks  should  be  realistic  and  should 
help  the  students  see  the  big  picture.  Successful  transfer  of  digital  skills  to  operational 
contexts  is  a  result  of  training  that  provides  specific  examples  of  when  and  how  certain 
procedures  will  be  used  while  performing  one’s  job  duties  (e.g.,  Baldwin  &  Ford,  1988; 

Clark  &  Vogel,  1985). 

a.  The  digital  skills  cd-based  tool  encourages  instructors  to  think  of  the  whole  task  (not 
individual  lessons)  and  think  of  alternative  approaches  to  the  Guided  Demonstration- 
PE-Exam  method. 

b.  One  way  to  obtain  realistic  operational  examples  and  develop  new  PEs  would  be  to 
talk  to  Soldiers  and  leaders  who  have  been  deployed  and  perhaps  residential  BCTC 
instructors. 

c.  Even  “buttonology”  could  be  put  in  an  operational  context  -  e.g.,  “When  this 
situation  happens,  you  do  the  following. . .”  This  would  help  the  students  to 
understand  why  and  how  FBCB2  is  used  in  the  real  world,  with  an  emphasis  on  the 
tactical  situation  and  its  implications. 

d.  Soldiers  will  be  more  motivated  to  learn  if  they  can  see  the  benefits  of  mastering  the 
knowledge  or  skill  and  how  it  can  be  applied  on  the  job  after  the  training  (Abell, 
2000). 

4.  Use  PEs  to  foster  problem-centered  learning.  As  noted  in  the  introduction,  more 
experienced  students  may  learn  digital  skills  best  if  they  can  shape  their  own  learning 
experiences.  Leibrecht  et  al.  (2007)  suggested  that  even  students  who  are  novices  at  the 
beginning  of  the  course  possess  enough  skill  by  the  end  of  the  course  to  benefit  from  PEs  that 
encourage  them  to  learn  the  system  on  their  own.  Further,  problem-centered  (constructivist) 
instructional  techniques  have  been  shown  to  be  superior  to  guided  demonstration 
instructional  techniques  because  students  are  encouraged  to  integrate  prior  tasks  and  skills 
with  newly  learned  ones.  This  helps  students  to  master  an  ever  increasing  skill  set. 

a.  PEs  can  be  developed  that  encourage  the  students  to  think  how  they  would  employ 
the  system  without  first  being  shown  step-by-step  how  to  do  it.  Often  the  PEs  simply 
repeat  what  the  instructor  did  and  stop  there.  But  that  is  only  the  first  step  in  student 
learning  and  does  not  guarantee  that  the  student  will  retain  the  information  or  be  able 
to  apply  the  function  in  a  different  context  (on  the  battlefield).  PEs  could  be 
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developed  that  provide  little  guidance  to  the  students  so  that  they  are  encouraged  to 
solve  novel  problems  on  their  own. 

b.  Instructors  should  think  of  operational  situations  that  will  require  users  to  perform 
certain  tasks  -  this  will  help  students  to  leam  not  only  “how”  to  perform  tasks  but 
also  “when”  to  perform  these  tasks. 

c.  Performing  the  tasks  in  multiple  practice  scenarios  may  help  the  students  to  apply  the 
skills  in  different  contexts  when  deployed. 

d.  Instructors  may  need  an  inventory  of  PEs  so  that  they  can  adapt  to  specific  classes  by 
matching  the  scenarios  to  the  backgrounds  and  experiences  of  the  students. 

e.  Some  research  suggests  that  individuals  do  not  learn  from  the  experience  per  se  but 
from  reflecting  on  the  experience.  Students  should  be  given  some  time  after  each  PE 
to  evaluate  and  discuss  those  learning  strategies  that  worked  well,  those  that  did  not 
work  so  well,  and  why  (Abell,  2000). 

5.  Leverage  observe-only  demonstration.  The  technique  of  “demonstration”  where  the  students 
watch  the  instructor  prior  to  performing  the  steps  themselves  may  help  them  to  retain  the 
information  better  than  “guided  demo”  where  they  perform  the  steps  for  the  first  time  along 
with  the  instructor.  In  guided  demo,  the  students  have  to  split  their  attention  which  makes  it 
more  difficult  for  them  to  think  through  the  steps  which  is  a  key  to  remembering  them. 

6.  Expand  practice  opportunities.  Procedural  knowledge,  or  the  ability  to  perform  a  sequence 
of  steps  in  a  competent  manner,  is  only  acquired  through  consistent  and  extensive  practice 
(Ackerman,  2006).  As  many  of  the  FBCB2  tasks  involve  a  sequence  of  steps,  one  practice 
session  per  topic  may  not  be  enough  for  the  students  to  retain  the  information  over  time. 
Multiple  practice  sessions  may  be  needed  for  critical  tasks.  Further,  as  the  course  presents  a 
lot  of  information  in  a  short  period  of  time,  additional  practice  opportunities  would  help  the 
students  to  become  familiar  with  the  content  and  reduces  the  cognitive  load  associated  with 
processing  unfamiliar  material  (Kirschner  et  al.,  2006). 

a.  However,  as  this  requires  additional  time,  the  instruction  of  other  course  topics  may 
have  to  be  curtailed.  Instructors  could  utilize  the  findings  of  Bink  et  al.  (2008)  to 
determine  which  FBCB2  functions  and  procedures  combat  veterans  used  the  most  in 
deployed  settings  and  allocate  more  practice  time  for  these  course  topics. 

7.  Emphasize  independent  practice.  Our  results  showed  that  one  dL  technique  involved  taking 
over  the  students’  computers  to  show  them  the  correct  sequence  of  steps.  Although  the 
instructors  may  have  to  use  this  technique  in  certain  situations,  the  students  who  are  having 
difficulty  may  not  retain  the  information  as  much  as  if  they  were  encouraged  to  practice  and 
leam  the  steps  on  their  own.  When  they  are  in  a  similar  situation  when  deployed,  they  may 
have  difficulty  remembering  the  procedures.  If  they  are  encouraged  to  learn  it  on  their  own, 
they  may  gain  self-confidence,  retain  the  skills  better,  and  be  better  able  to  work 
independently.  This  is  especially  important  for  those  students  who  are  taking  the  class  to 
train  other  Soldiers  in  their  units. 

a.  As  this  may  require  additional  classroom  time,  the  instructors  should  have  additional 
PEs  available  for  the  other  students  who  have  completed  the  initial  PE.  This  would 
provide  the  other  students  in  the  class  with  more  opportunities  for  practice. 


43 


b.  If  the  instructor  must  take  over  the  student’s  computer  to  show  the  correct  steps,  the 
student  should  immediately  be  given  the  opportunity  to  practice  the  steps  on  his/her 
own. 

c.  Helping  students  to  operate  the  system  independently  and  retain  the  skills  over  time 
may  mean  that  the  course  only  covers  critical  tasks. 

8.  Provide  ample  training  aids.  The  utility  of  student  handouts  is  especially  critical  in  the  dL 
environment.  The  physical  separation,  especially  in  the  absence  of  an  AI,  and  the  limited 
communication  links  make  it  more  difficult  to  clear  up  student  confusion. 

Conclusions 

Both  the  dL  instructors  and  students  acknowledged  substantial  value  in  the  VTT 
approach  to  training  digital  skills  and  offered  good  suggestions  for  improving  the  training  for  dL 
environments.  While  the  sampling  approach  was  limited,  the  results  provide  insights  and  lessons 
for  ensuring  effective  courses  for  ABCS  operators.  The  findings  contribute  to  the  Army’s  efforts 
to  enhance  blended  learning  solutions  to  meet  critical  training  needs.  Many  of  the 
recommendations  are  applicable  to  both  dL  and  traditional  classroom  settings,  and  we  encourage 
instructors  to  use  these  ideas  to  make  small  changes  to  their  training  that  may  result  in  big 
differences  in  their  students’  proficiency  levels. 
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APPENDIX  A 


Instructor  Observation  Form 


Part  I.  Class  Details  for  Instructor  Observations 


Data  Collection  Form 


Observer: 

Date: 


1.  Course:  Distributed  FBCB2 

2.  Location:  (Installation) _  (classroom#) _ 

3.  Software  version: _ 

4.  #  of  instructors: _  #  of  classroom  monitors: _  #  of  students: 

5.  #  of  workstations: _ 

6.  Describe  the  training  site  (to  include  resources;  draw  a  diagram  of  site  below). 


7.  Duration  of  course  (days/  hours) _ 

8.  Observation  period:  (from  day/hour) _ (to  day/hour) 

9.  Diagram  of  site: 


A-l 


A-2 


PART  II A:  Activity  Chronology 


ACTIVITY  CODE 

VID  -  video 

DEM  -  demonstration 

G/D  -  guided  demo 

PE  -  practical  exercise 

REV  -  review 

BRK  -  break 

TEST  -quiz,  exam,  etc 

LEC  -  lecture 

SHEET  NUMBER: 

Observer: _ 

Date: _ 


Training  Techniques 

MA-  uses  Memory  Aids 

RM  -  Relates  to  Military  operation 

EA-  Encourages  Active  learning 

PP  -  Purpose  and  Path 

RP  -  Relates  to  Previous  content 

EP  -  Emphasizes  Practice 

RG  -  Relates  to  General  knowledge 

SP-  points  out  unique  Screen  Prompts  or  cues 

LC  -  Learning  Check 

RL  -  Responds  to  Learners 

dL  -  Uses  techniques  specific  to  dL 

Num 

Activity 

Code 

Time 

Description  of  topics  covered  and  instructor/student  behaviors 

Training 

Techniques 

Part  IIB.  PE  Sheet 


Num 

Exercise  Type: 

Exercise 

How  did  instructor 

Peer 

#  Reps  of  Same 

•  Guided 

Context: 

monitor  progress? 

coaching 

Exercise 

Additional  Comments 

(from 

•  Same  as  Demo 

observed? 

Chronology) 

•  New  situation 

•  Military  Ops 

•  Asked  for  feedback 

•  Integrate  prior 

•  Job-relevant 

•  Observed  Soldiers 

Y/N 

tasks 

•  Arbitrary 

•  Assistants  observed 

Start-of-course  questions 


1.  What  level  of  proficiency  on  this  system  were  students  required  to  have  before  taking  this  course? 
(check  all  that  apply) 

_ No  pre-requisite  knowledge  of  the  system  was  required 

_ Students  had  to  understand  the  following  software  attributes  and/or  functions _ 


Students  had  to  have  completed  the  following  course(s): 


Other  (e.g.  exam)  describe: 


2.  At  the  beginning  of  the  course,  how  did  the  instructor  communicate  the  training  objectives  (i.e.  topics 
or  skills  to  be  covered)?  List  those  objectives  if  they  are  not  listed  in  your  chronology. 


3.  At  the  beginning  of  the  course,  what  were  students  told  to  expect  for  a  final  test? 


Final  Assessment  Description 


1 .  How  did  the  instructor  evaluate  student  proficiency  at  the  end  of  the  training  session?  (e.g.,  knowledge 
(exam),  performance  (PE),  etc.) 

a.  _ Test  of  knowledge  (written  exam) 

_ Recall  (students  repeated  what  was  taught  in  class) 

_ Application  (students  applied  what  was  taught  to  solve  problems) 

_ Other  (describe: _ ) 

b.  _ Test  of  performance/skill  (PEs) 

_ Recall  (students  repeated  tasks  taught  in  class) 

_ Application  (students  applied  what  was  taught  to  solve  problems) 

_ Other  (describe: _ ) 

c.  _  There  was  no  evaluation  of  proficiency 


Describe  the  PE/Test  and  Feedback  (if  possible,  obtain  a  copy  of  the  exam): 
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APPENDIX  B 


BCTC-DODGE:  FBCB2  Instructor  Interview  Questions 


1 .  Have  you  used  FBCB2  in  an  operational  setting?  Please  provide  specific  details  of  how  you 
have  used  FBCB2  in  an  operational  setting. 

2.  Have  you  taught  FBCB2  in  a  residential  setting?  Approximately  how  many  times? 

3.  Approximately  how  many  times  have  you  taught  FBCB2  in  the  distributed  format?  Other 
than  at  BCTC? 

4.  Have  you  taught  any  of  the  other  ABCS  systems  in  a  distributed  learning  format? 
Approximately  how  many  times?  Other  than  at  BCTC? 

5.  Have  you  had  formal  training  as  a  dL  instructor?  If  so,  when  and  where  did  the  training 
occur?  Do  you  think  it  prepared  you  to  teach  FBCB2  in  a  distributed  format? 

6.  Are  there  specific  instructional  design  or  learning  principles  that  you  follow  when  developing 
your  instruction ? 

a.  Prompts:  Instructional  design  techniques:  e.g.,  scaffolding  (instructor  models  the 
desired  learning  strategy  or  task,  then  gradually  shifts  responsibility  to  the 
students),  advanced  organizers  (instructor  provides  ways  to  help  students  link 
their  ideas  with  new  material  or  concepts),  GEL  (guided  experiential  learning  - 
specific  sequence  of  demonstration,  practice,  and  evaluation) 

7.  Are  there  specific  instructional  design  or  learning  principles  that  you  follow  when  preparing 
to  teach/present  the  FBCB2  course? 

a.  Prompts:  Instructional  design  techniques:  e.g.,  scaffolding  (instructor  models  the 
desired  learning  strategy  or  task,  then  gradually  shifts  responsibility  to  the 
students),  advanced  organizers  (instructor  provides  ways  to  help  students  link 
their  ideas  with  new  material  or  concepts),  GEL  (guided  experiential  learning  - 
specific  sequence  of  demonstration,  practice,  and  evaluation) 

8.  What  techniques  for  teaching  FBCB2  have  you  found  to  be  helpful  in  facilitating  the 
students’  learning  of  the  system? 

a.  Prompts:  Various  instructional  activities,  conducting  learning  checks,  providing 
feedback  to  students,  interacting  with  students,  measuring  performance, 
conducting  PEs,  motivating  students 

b.  If  the  instructor  has  taught  a  resident  ABCS  course...  How  do  these  techniques 
differ  from  teaching  a  residential  ABCS  course? 

9.  What  techniques  for  teaching  FBCB2  have  you  found  to  be  helpful  in  keeping  the  students’ 
all  on  the  same  pace  as  your  instruction? 

a.  If  the  instructor  has  taught  a  resident  ABCS  course...  How  do  these  techniques 
differ  from  teaching  a  residential  ABCS  course? 

10.  What  techniques  for  teaching  FBCB2  have  you  found  to  be  helpful  in  answering  students’ 
questions  regarding  specific  steps  or  procedures? 
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a.  If  the  instructor  has  taught  a  resident  ABCS  course...  How  do  these  techniques 
differ  from  teaching  a  residential  ABCS  course? 

1 1 .  What  techniques  for  teaching  FBCB2  have  you  found  to  be  helpful  in  working  with  a  student 
who  is  having  much  difficulty  with  specific  steps  or  procedures? 

a.  If  the  instructor  has  taught  a  resident  ABCS  course...  How  do  these  techniques 
differ  from  teaching  a  residential  ABCS  course? 

12.  If  the  instructor  has  taught  a  resident  ABCS  course...  Do  you  think  the  dL  approach  is  more 
or  less  effective  for  learning  the  system? 

a.  Prompts:  student/teacher  ratio,  ability  to  take  over  students’  machines,  different 
techniques  used,  different  military  population 

13.  Are  there  any  specific  advantages  or  disadvantages  for  teaching  FBCB2  in  a  distributed 
format? 

14.  Flave  you  received  feedback  from  students  regarding  the  FBCB2  course  -  positive, 
negative? 

a.  Do  you  modify  the  FBCB2  course  or  instructional  approaches  based  on  student 
feedback? 

1 5.  Do  you  think  that  students  retain  the  digital  skills  for  FBCB2  over  time?  Do  you  know 
whether  they  receive  refresher  training  on  FBCB2? 

16.  Flow  well  does  the  available  network,  remote  instructional  capabilities,  and  interactive/ 
collaborative  capabilities  support  teaching  distributed  ABCS  courses? 

17.  What  do  you  like  best  about  teaching  the  distributed  FBCB2  course? 

18.  What  do  you  like  least  about  teaching  the  distributed  FBCB2  course? 

19.  What  problems  do  you  typically  encounter  during  the  course  that  affect  the  way  you  present 
the  material  or  the  degree  to  which  the  students  learn  the  system? 

20.  What  ideas  do  you  have  for  improving  the  distributed  FBCB2  course? 

21 .  Any  other  comments  and  suggestions? 
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Part  II:  Activity  Chronology 


Student  Behaviors 

OB  -  Other  student  Behaviors  are  observed 

SM  -  Instructor  takes  over  Student’s  Mouse 

AR  -  Asks  instructor  to  Repeat  step 

NP  -  Multiple  students  do  Not  Pay  attention  to 
the  lecture/demonstration 

UO  -  Uses  Other  functions  than  those  for  current  task 

AQ  -  Asks  instructor  a  Question 

PA  -  Pays  attention  to  lecture/PowerPoint 
presentation/Demonstration 

WF  -  Multiple  students  Waits  for  others  to  Finish 
task/PE 

GS  -  Works  in  Groups  to  Solve  individual  tasks 

PC  -  Peer  Coaching  (Seeks/Gives  help) 

WA  -  Works  Ahead  of  instruction  but  on  same  task 

IS  -  Works  Independently  to  Solve  problems 

RQ  -  Responds  to  instructor  Questions 

LB  -  Lags  Behind  instruction 

SC  -  Side  Conversations  during  instruction 

Num  =  Number  the  classroom  activities  (Activity 
Codes)  sequentially 

Activity  Code  =  Use  the  codes  at  the  top  of  the  page 
to  indicate  the  instructional  activity 

Time  =  Record  student  behaviors  for  every  5- 
minute  block  of  instruction 

Descriptions  =  Capture  examples  of  behavior; 
if  OB  is  used  please  describe  behavior  in  detail 

Student  Behaviors  =  Use  codes  of  student  behaviors; 

If  behavior  is  observed  once  in  a  5 -minute  period  then 
record  the  behavior;  If  multiple  students  are  observed 
performing  the  behavior  then  note  that  in  the 

Description  column. 

Num 

Activity 

Code 

Time 

Description  of  topics  covered  and  instructor/student  behaviors 

Student 

Behaviors 

ACTIVITY  CODE 

VID  -  video 

DEM  -  demonstration 

G/D  -  guided  demo 

PE  -  practical  exercise 

REV  -  review 

BRK  -  break 

TEST  -quiz,  exam,  etc 

LEC  -  lecture 

SHEET  NUMBER: 

Date: _ 

#  of  Students: _ 


Part  III:  Summary  Questions 


Summary  Questions,  (to  be  answered  after  each  observation  period). 

1 .  What  percent  of  this  course  covered  operator  (i.e.  knobology)  skills  and  what  percent  covered 
employment  skills? 


2.  Was  the  material  taught  at  an  appropriate  level  of  difficulty  for  the  students?  Did  the  students  appear 
to  be  engaged? 


3.  Were  there  enough  systems  for  all  the  students?  Did  hardware  or  software  problems  occur  and  if  so 
did  they  impede  progress  in  the  class? 


4.  Were  there  any  innovative  teaching  techniques  that  deserve  special  mention?  If  so,  describe  them. 
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APPENDIX  D 


Student  Survey 

BIOGRAPHICAL  INFORMATION 

1.  Today’s  Date: _ 

2.  Rank/Grade: _  3.  Branch: _  4.  AOC/MOS _ 

Time  in  Service  as:  5a.  Enlisted _ yrs _ mos  5b.  Officer _ yrs _ mos 

6.  Status  (circle  one):  Active  Duty  Army  Reserve  National  Guard 

7.  Unit: _  8.  Current  Duty  Position: _ 

9.  Time  in  Current  Duty  Position: _ mos 


10.  Military  Education  (Check  all  that  apply) 


OES 

OCS/ROTC 

OBC/BOLC  III 

OAC/CPTs  Career  Course 

PCC 

Other 

NCOES 

PLDCAVLC 

BNCOC 

ANCOC 

1SG  Course 

Other 

11.  Deployment  Experience  (Provide  information  for  the  last  5  years) 


Position(s) 

Total  Months 

OIF 

OEF 

Other 

Other 
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12.  Digital  Training  (Provide  information  for  all  that  apply) 


System 

Training  Course  (Type,  Month/Year  of 
Training,  #  hrs) 

Individual  Training  within  Unit  (Type, 
Average  #  hrs  used  per  week) 

FBCB2 

ASAS 

AFATDS 

MCS 

Other 

13.  Digital  Experience  (Provide  information  for  all  that  apply) 


System 

Used  in  Field  Training  (Average  #  hrs 
used  per  week) 

Used  in  Deployment  Ops  (Average  #  hrs 
used  per  week) 

FBCB2 

ASAS 

AFATDS 

MCS 

Other 

14.  How  comfortable  are  you  using  a  computer?  (Circle  one) 

O-Not  at  All  1-Slightly  Comfortable  2-Moderately  Comfortable  3-Very  Comfortable 

15.  How  much  experience  do  you  have  using  a  computer?  (Circle  one) 

0-No  Experience  1 -Little  Experience  2-Moderate  Experience  3-Much  Experience 

16.  Why  are  you  attending  this  FBCB2  training  course? 
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FBCB2:  Distributed  Learning  Survey 


1)  After  taking  the  FBCB2  course,  to  what  extent  do  you  feel  that  you  learned  the 
system? 

□  I  did  not  learn  much  at  all 

□  I  partially  learned  the  system 

□  I  learned  all  steps  and  processes  of  the  system 

□  I  became  proficient  and  could  teach  the  system  to  someone  else 


2)  To  what  extent  do  you  feel  that  the  distributed  format  (instructor  not  located  in  the 
same  room  as  you)  affected  your  learning  of  FBCB2? 

□  Hindered  learning  greatly 

□  Hindered  learning  somewhat 

□  Did  not  hinder  nor  facilitate  learning 

□  Facilitated  learning  somewhat 

□  Facilitated  learning  greatly 


Please  provide  specific  examples  of  how  the  distributed  format  either  helped  or 
hindered  your  learning  of  FBCB2. 


3)  To  what  extent  was  the  FBCB2  course  more  or  less  effective  than  other  distributed 
learning  courses  you  have  participated  in. 

□  I  have  not  attended  other  distributed  learning  courses 

□  Much  less  effective  than  other  distributed  learning  courses 

□  Somewhat  less  effective  than  other  distributed  learning  courses 

□  About  the  same 

□  Somewhat  more  effective  than  other  distributed  learning  courses 

□  Much  more  effective  than  other  courses 

Please  explain  why  this  FBCB2  course  was  more  or  less  effective  in  helping  you  to 
learn  than  other  distributed  courses.  Please  provide  specific  examples  of  differences. 


4)  To  what  extent  was  the  FBCB2  course  more  or  less  effective  in  helping  you  learn  the 
system  than  resident  ABCS  courses. 

□  I  have  not  attended  any  resident  ABCS  courses 

G  Much  less  effective  than  other  resident  ABCS  courses 

□  Somewhat  less  effective  than  other  resident  ABCS  courses 
G  About  the  same 

G  Somewhat  more  effective  than  other  resident  ABCS  courses 

□  Much  more  effective  than  other  resident  ABCS  courses 

Please  explain  why  the  distributed  format  was  more  or  less  effective  in  helping  you 
learn  FBCB2  compared  to  resident  ABCS  courses. 
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5)  Please  use  the  following  scale  to  rate  the  difficulty  or  ease  of  learning  the  different 
FBCB2  functions  or  procedures  that  may  have  been  instructed  during  the  course. 
Please  check  the  appropriate  box. 


FBCB2 

Functions 

and 

Procedures 

Very 

Difficult 

Somewhat 

Difficult 

Neither 

Difficult 

nor 

Easy 

Somewhat 

Easy 

Very 

Easy 

Prep  for  Combat  Configure  Role; 
Clear  Logs  and  Queues,  etc 

Messages 

Prepare/send  Cbt  Msg;  Spot  Report; 
Addressing,  etc 

Navigation  Tools 

CLOS;  LOS;  Periodic  Reminders,  etc 

Orders 

Prepare  and  send  Warno/Opord; 
Attach  Overlay,  etc 

Overlay  Messages 

Create  Objects;  Save/Send  Overlay, 
etc 

Please  provide  comments  to  explain  why  you  had  difficulty  learning  a  specific 
function/task. 


Please  provide  comments  to  explain  why  you  found  it  easy  to  learn  a  specific 
function/task. 
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6)  Please  use  the  following  scale  to  rate  the  effectiveness  of  different  instructional 
activities  that  may  have  been  used  during  the  course.  Please  check  the  appropriate 
box. 


Instructional  Activities 

Very 

Ineffective 

Somewhat 

Ineffective 

Neither 

Ineffective 

or 

Effective 

Somewhat 

Effective 

Very 

Effective 

Demonstration  of  steps  and 
procedures  without  the 
students  using  the  system  to 
follow  along 

Guided  demonstration  of 
steps  and  procedures  with 
the  students  using  the  system 
to  follow  along 

Lecture  /  PowerPoint  slides 

Practical  Exercise 

Video 

Review  of  material  that  was 
covered 

Test  -  quiz,  exam 

Given  that  the  instructor  teaches  this  course  from  a  remote  location,  please  describe 
some  instructional  techniques  that  would  have  helped  you  learn  FBCB2  better. 


7)  Please  indicate  how  you  feel  about  the  pace  of  instruction  for  you. 

O  Extremely  slow 

□  Somewhat  slow 

□  Just  Right 

□  A  little  fast-paced 

□  Extremely  fast-paced 

Please  provide  examples  of  times  when  you  felt  the  instruction  was  too  fast-paced 
and  recommendations  for  how  the  instructor  can  keep  everyone  on  the  same  pace. 
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8)  To  what  extent  did  you  work  ahead  of  the  FBCB2  instruction? 

□  Never 

□  Only  during  certain  topics 

□  Throughout  the  entire  course 

If  you  worked  ahead  of  the  instruction,  please  explain  whether  the  instruction  was  too 
slow  or  whether  you  have  experience  with  using  FBCB2. 

If  you  have  experience  using  FBCB2,  what  would  make  the  instruction  more 
interesting  for  you? 

9)  To  what  extent  did  you  seek  assistance  from  your  fellow  classmates  regarding  certain 
steps  or  procedures  for  FBCB2? 

□  Never 

□  Only  during  certain  topics 

□  Throughout  the  entire  course 

If  you  asked  your  peers  for  assistance,  were  they  able  to  help  you  to  understand  the 
steps  or  specific  procedures  involved  in  the  task?  What  methods  did  they  use  that 
helped  you  understand  the  procedures? 

10)  To  what  extent  did  you  assist  other  students  in  learning  certain  steps  and  procedures 
of  FBCB2? 

□  Never 

□  Only  during  certain  topics 

□  Throughout  the  entire  course 

If  you  helped  other  students,  please  describe  which  procedures  you  assisted  with 
and  the  methods  you  used  to  help  your  classmates  learn  the  steps. 

11)  To  what  extent  was  the  instructor  able  to  answer  all  of  your  questions  and  provide 
clarification  for  certain  FBCB2  steps  and  procedures? 

□  Answered  no  questions  /  Clarified  no  procedures 

□  Answered  some  but  not  all  questions  /  Clarified  some  but  not  all  procedures 

□  Answered  all  questions  /  Clarified  all  procedures 

If  you  felt  that  the  instructor  did  not  provide  clarification,  do  you  think  that  this  was  a 
function  of  the  distributed  format  (i.e.,  would  your  questions  have  been  answered  if 
you  were  able  to  discuss  them  face-to-face  with  the  instructor)? 

Please  explain  how  this  affected  your  learning  of  FBCB2. 
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12)  To  what  extent  were  you  engaged  and  motivated  to  learn  throughout  the  FBCB2 
course? 

□  Not  engaged/motivated  at  all 

□  Somewhat  engaged/motivated 

□  Very  engaged/motivated 

□  Extremely  engaged/motivated 

Please  identify  which  factors  influenced  your  motivation  level,  to  include  the 
distributed  learning  environment. 


Please  provide  examples  of  the  types  of  things  that  the  instructor  did  to  hold  your 
interest  in  the  class? 


13)  What  did  you  like  the  best  about  the  distributed  learning  course? 

14)  What  did  you  like  least  about  the  distributed  learning  course? 

15)  What  problems  did  you  encounter  during  the  course? 

16)  How  would  you  improve  the  distributed  learning  course? 

17)  Any  other  comments  and  suggestions? 
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APPENDIX  E 


Instructions  for  Observations 
Instructor  Observer  Instructions 


BACKGROUND:  The  purpose  of  this  project  is  to  observe,  record,  and  report  distributed  training 
techniques  and  the  possible  effects  of  those  techniques  on  student  learning  of  digital  skills.  A  key 
objective  of  this  project  is  to  record  instructor  behaviors  that  occur  in  a  distributed  learning  environment. 
As  you  observe  the  instructor  please, 

>  Capture  any  and  all  information  relevant  to  the  project  objective: 

“Identify  learning  principles  in  use  and  how  to  improve  them.” 

>  Observe  -  listen  -  record 

>  Record  objective  data;  note  if  an  entry  is  based  on  opinion.  We  will  assess  the  data  later. 

>  Be  unobtrusive;  do  not  interfere. 

>  Record  as  much  detail  as  possible;  when  in  doubt,  write  it  down. 

(We  should  be  able  to  reconstruct  a  mental  picture  of  what  occurred.) 

»  The  OBSERVATION  FORM  is  divided  into  3  parts: 

•  Part  I,  Class  Details 

•  Part  II,  Instructional  Activities  and  Practical  Exercises 

•  Part  III,  Start-of  Course  and  Final  Assessment 

Instructions  for  completing  each  section  are  below. 

Part  I,  Class  Details.  The  first  page  of  the  data  collection  instrument  asks  for  details  about  the 
classroom.  The  questions  are  self  explanatory. 

Part  II,  Instructional  Activities  and  Practical  Exercises.  Once  the  classroom  instruction  begins,  there 
are  two  forms  for  recording  instructional  activities:  the  Activity  Chronology  form  and  the  PE/Guided  Demo 
Detail  Sheet. 

IIA,  Activity  Chronology.  Use  this  form  to  record  instructional  activities  as  they  occur,  as  follows: 

Column  1  (Num)  -  Number  the  classroom  activities  sequentially  on  the  chronology  (use  additional  copies 
of  the  chronology  page  as  needed).  When  the  Demo  Detail  sheet  is  used,  the  activity  number  from  the 
Chronology  sheet  should  be  entered  in  the  first  (Num)  column  of  the  Demo  Detail  Sheet. 

Column  2  (Activity  Code)  -  Use  the  codes  at  the  top  of  the  page  to  indicate  the  type  of  activity.  If  more 
than  one  line  is  needed  to  describe  the  activity,  there  is  no  need  to  write  the  activity  code  for  each  line. 

Column  3  (Time)  Note  the  time  that  begins  each  5  minute  observation  period  (see  details  of  time¬ 
sampling  observation  procedure  below).. 

Column  4  (Description/Topics  covered)  -  This  space  should  be  used  for  detailed  notes  about  the  course 
content  and  student  and  instructor  behaviors.  Detail  is  important;  it’s  better  to  write  down  too  much  than 
too  little.  Use  this  space  to: 

•  Describe  the  topic  or  task  being  covered  as  defined  by  the  instructor;  note  the  steps. 

•  Note  examples  of  the  checks  on  learning  employed. 

•  Capture  examples  of  the  learning  principles. 

•  Provide  examples  of  the  student/student  or  instructor/student  coaching  or  tutoring  that  takes 
place. 

•  Explain  any  linkage  between  the  current  activity/task  and  previous  activities/tasks. 

•  Note  if  some  steps/points  had  to  be  presented  multiple  times. 

•  Capture  clarifying  questions  asked  by  students. 

•  Record  other  pertinent  information  that  will  assist  in  understanding  what  transpired  during  the 
activity. 
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Column  5  (Learning  Principles)  -  Using  the  abbreviations  in  the  table  at  the  top  of  the  chronology  page, 
enter  all  relevant  learning  principles  exemplified  by  the  activity  described  on  that  line  of  the  chronology. 

The  learning  principles  are  described  in  more  detail  below: 

•  Points  out  screen  prompts  or  cues,  to  guide  responses:  The  instructor  points  out  flashing 
numbers,  grayed  out  buttons,  screen  text,  etc.  that  cue  the  student  about  what  action  is  needed. 

•  Emphasizes  practice:  The  instructor  indicates  the  importance  of  practicing  the  tasks  to  gain  the 
required  skill. 

•  Responds  to  learner  statements:  The  instructor  modifies  the  instruction  in  some  way  to 
accommodate  a  question  or  request  from  a  student. 

•  Encourages  active  learning:  In  addition  to  PEs,  the  instructor  prods  students  to  actively  process 
the  material.  For  example  by  asking  questions,  giving  students  problems  to  solve  individually  or 
in  small  groups,  or  having  students  answer  each  others’  questions. 

•  Presents  purpose  and  path:  The  organization  of  the  class  material  is  made  explicit  to  students 
via  a  table  or  graphic  and/or  an  effort  is  made  to  explain  the  reason  for  the  organization. 

Students  are  reminded  of  this  organization  and  where  they  are  in  the  order  of  topics  throughout 
the  class. 

•  Relates  to  military  operation:  The  instructor  relates  the  material  to  military  operations  or 
knowledge  either  by  making  an  analogy  (“Overlays  function  just  like  acetate  overlays  on  a  paper 
map’)  or  by  describing  how  the  software  might  be  used  during  a  military  operation  (“The  next  time 
you  have  to  plan  a  convoy  route,  remember  to  use  the  CLOS  tool.’). 

•  Relates  to  general  knowledge:  The  instructor  relates  the  material  to  general  knowledge  about 
computers,  etc.  “ This  is  similar  to  typical  e-mail,  operates  like  normal  Windows  functions.”) 

•  Relates  to  previous  content:  The  instructor  relates  the  material  to  some  previously  covered  topic 
or  exercise. 

•  Uses  memory  aids:  The  instructor  provides  memory  aids  (such  as  PACS  [PLGR,  Antenna, 
Computer,  Screen]  to  remember  startup  sequence  for  FBCB2)  or  related  techniques. 

MB,  PE/Guided  Demo  Detail  Sheet.  Use  this  form  to  record  information  during  PEs  (Practical  Exercises) 
or  Guided  Demos.  The  fields  of  this  form  are  described  below. 

Column  1  (Num)  -  Use  this  to  record  the  activity  number.  This  number  should  match  the  activity  on  the 
Num  column  of  the  Activity  Chronology  form.  If  multiple  PEs  or  demos  are  conducted  for  a  single 
“Activity”,  then  list  each  separately  on  the  PE/Guided  Demo  detail  using  la,  1b,  1c,  etc. 

Column  2  (Exercise  Type)  -  Indicate  the  nature  of  the  exercise.  Options  progress  from  easy  to  difficult. 

•  Guided  means  students  repeat  the  actions  of  the  instructor  by  following  along  step  by  step. 

•  Same  as  Demo  means  that  after  watching  a  Demo,  students  must  repeat  all  the  steps  on  their 
own. 

•  New  Situation  means  that  after  watching  a  demo,  students  must  repeat  the  steps  but  enter  new 
information  in  some  of  the  fields  (e.g.,  different  addressees,  different  SPOT  contents,  new 
overlay). 

•  Integrated  Prior  Tasks  means  that  students  must  combine  knowledge  of  tasks  or  information 
learned  at  different  times  during  the  class  in  order  to  complete  the  PE  successfully. 

Column  3  (Exercise  Context)  -  Indicate  whether  an  attempt  is  made  to  relate  the  exercise  to  a  military 
operation,  a  job-relevant  task  (not  necessarily  associated  with  an  operation)  or  some  arbitrary  action  like 
“send  a  free-text  message  with  your  name.” 

Column  4  (How  did  instructor  monitor  progress?)  -  Describe  the  means  used  to  monitor  student  progress. 
Column  5  (Peer  coaching  observed?)  -  Indicate  whether  or  not  you  observed  peer  coaching/mentoring. 
Column  6  (#  Reps  of  Same  Exercise)  -  Tally  how  many  times  the  exact  exercise  was  repeated. 

Column  7 (Additional  comments)  -  Use  this  space  to  record  any  other  relevant  information. 
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Time-Sampling  Procedure:  To  simplify  the  quantification  of  learning  principles,  we  will  use  a  time¬ 
sampling  procedure  in  which  we  will  note  whether  or  not  learning  principles  occurred  within  5  minute 
intervals.  During  any  given  5  minute  interval,  record  a  description  of  the  content  being  covered  and  any 
relevant  student  and  instructor  behaviors  in  column  4.  In  column  5  (this  can  be  done  in  real-time  or  even 
at  a  later  time)  note  which  learning  principles  were  exemplified  by  the  behaviors  in  each  row  of  column  4 
(see  example  Activity  Chronology  below). 

Using  this  sampling  procedure,  you  can  aggregate  instances  of  each  learning  principle.  For  example  if  an 
instructor  spends  10  minutes  in  a  guided  demo  explaining  how  to  send  an  overlay,  you  would  describe 
the  content  of  what  he  was  teaching  but  rather  than  tally  the  number  of  screen  prompts  he  indicates  you 
would  simply  note  “SP”  in  the  5th  column  for  each  5  min  interval  that  the  instructor  was  doing  the  guided 
demo.  Likewise  you  would  note  any  other  learning  principles  observed  and  ideally  each  learning  principle 
would  have  a  description  in  the  same  row  of  column  4. 

You  may  note  more  than  one  instance  of  any  learning  principle  during  a  five  minute  interval  especially  if 
they  are  discrete  and  separated  by  other  activities.  For  example,  an  instructor  may  end  one  guided  demo 
and  then  spend  a  couple  of  minutes  answering  questions  and  then  begin  another  guided  demo  within  one 
5  min  observation  interval.  Go  ahead  and  note  “SP”  when  the  first  guided  demo  ends  and  then  “LC” 
(learning  check)  as  the  instructor  asks  questions  and  then  “SP  as  the  next  guided  demo  begins.  It’s  okay 
to  record  learning  principles  at  a  higher  level  of  precision  than  we  will  actually  report  them. 

If  an  activity  spans  across  two  time  intervals,  for  example  if  a  guided  demo  lasts  more  than  5  minutes,  you 
would  briefly  describe  the  guided  demo  in  each  interval  that  it  occurs  (using  ditto  marks  is  acceptable  as 
long  it’s  clear  what  they  refer  to)  and  also  indicate  any  learning  principle  observed  during  each  5  minute 
interval  that  the  guided  demo  spans. 

You  only  need  to  complete  one  row  of  sheet  MB  (PE  Guided  demo  detail  sheet)  for  each  guided 
demo/PE. 

To  help  clarify  this  procedure,  see  the  example  Activity  Chronology  below. 

Part  III,  Start-of-Course  and  Final  Assessment.  There  are  two  groups  of  questions  in  part  3.  The  start- 
of-course  questions  should  be  answered  when  observing  the  beginning  of  the  course.  The  final 
assessment  questions  only  pertain  to  the  assessment  given  at  the  end  of  the  course. 

It  may  be  useful  to  talk  briefly  with  the  instructor  just  before  the  class  begins  to  learn  things  like  the 
learning  objectives  of  the  class  or  how  the  instructor  intends  to  assess  proficiency  at  the  end  of  the  class. 
Otherwise  most  of  these  questions  should  be  addressed  in  the  instructor’s  opening  comments 
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IIA.  Activity  Chronology 


ACTIVITY  CODE 

VID  -  video 

DEM  -  demonstration 

G/D  -  guided  demo 

PE  -  practical  exercise 

REV  -  review 

BRK  -  break 

TEST  -quiz,  exam,  etc 

LEC  -  lecture 

SHEET  NUMBER: _ 1 

Observer: _ Jones 

Date:  9-Feb,  2006 


LEARNING  PRINCIPLES 

MA  -  uses  Memory  Aids 

RM  -  Relates  to  Military  operation 

EA-  Encourages  Active  learning 

PP  -  Purpose  and  Path 

RP  -  Relates  to  Previous  content 

EP  -  Emphasizes  Practice 

RG  -  Relates  to  General  knowledge 

SP-  points  out  unique  Screen  Prompts  or  cues 

LC  -  Learning  Check 

RL  -  Responds  to  Learners 

dL  -  Uses  techniques  specific  to  dL 

Num 

Activity 

Code 

Time 

Description  of  topics  covered  and  instructor/student  behaviors 

Learning 

Principles 

1 

LEC 

0815 

Reviewed  what  they  would  cover  for  the  day  and  reminded  them  of  upcoming  PE 

PP 

told  students  to  power  up  systems,  answered  questions  from  students 

RL 

2 

G/D 

0820 

Inst  provided  steps  to  build  3D  maps.  Two  students  asked  clarifying  questions 

SP,  RL 

0825 

cont  G/D  to  3D  maps.  Stopped  multiple  times  to  answer  student  questions,  appears 

SP,  RL 

to  be  a  problem  with  workstations.  No  AI  is  present  so  instructor  is  troubleshooting 

RL 

mm 

o 

OO 

co 

o 

nobody  can  build  3D  map  so  instructor  just  demonstrated.  Explained  operational  uses 

SP,  RM 

w 

Began  a  G/D  to  locate  and  select  a  map.  Inst  had  to  help  some  student  properly  configure 

SP,  RL 

their  machines 

0835 

continued  G/D  on  selecting  a  map 

SP 

0840 

continued  G/D.  stopped  to  repeat  steps  for  a  student.  Reminded  students  there  are 

SP,  RL 

multiple  ways  to  accomplish  this  task.  Asked  students  to  describe  alternatives. 

RP,  LC 

5 

G/D 

0845 

G/D  on  selecting  “Battle  Mode"  and  book  marking  maps.  Inst  notes  activities  2-5  are 

SP,  RP 

a  review  of  previous  day's  work. 

0850 

continued  G/D.  1  student  asked  a  question  about  sending  bookmarks 

SP,RL 

6 

G/D 

0855 

clean  up  system  to  prepare  for  PE.  Used  dual  screens  (PPTslides  vs.  System) 

SP,PP 

Student  Observer  Instructions 


INSTRUCTIONS  TO  STUDENT  OBSERVER 


BACKGROUND:  The  purpose  of  this  project  is  to  observe,  record,  and  report  distributed  training 
techniques  and  the  possible  effects  of  those  techniques  on  student  learning  of  digital  skills.  A  key 
objective  of  this  project  is  to  record  student  behaviors  that  occur  in  a  distributed  learning  environment.  As 
you  observe  the  students  please, 

>  Capture  any  and  all  information  relevant  to  how  students  respond  to  the  instruction  -  these 
behaviors  could  take  many  forms  such  as  working  in  groups  to  complete  individual  tasks, 
performing  behaviors  not  related  to  the  instruction,  or  performing  behaviors  to  show  that  they  are 
actively  engaged  in  the  instruction. 

>  Observe  -  listen  -  record 

>  Record  objective  data;  note  if  an  entry  is  based  on  opinion.  We  will  assess  the  data  later. 

>  Be  unobtrusive;  do  not  interfere.  However,  it  is  appropriate  to  walk  around  the  room  or  look  at 
individual  computer  screens  to  view  the  students’  activity. 

>  Record  as  much  detail  as  possible;  when  in  doubt,  write  it  down. 

(We  should  be  able  to  reconstruct  a  mental  picture  of  what  occurred.) 

»  The  OBSERVATION  FORM  is  divided  into  3  parts: 

•  Part  I,  Class  Details 

•  Part  II,  Student  Observation  Form 

•  Part  III,  Summary  Questions. 

Instructions  for  completing  each  section  are  below. 

Part  I,  Class  Details.  The  first  page  of  the  data  collection  instrument  asks  for  details  about  the 
classroom.  The  questions  are  self  explanatory. 

Part  II,  Student  Observation  Form.  Once  the  classroom  instruction  begins,  please  use  this  form  to 
record  the  student  behaviors  as  follows: 

Column  1  (Num)  -  Number  the  classroom  activities  sequentially  on  the  chronology  (use  additional  copies 
of  the  chronology  page  as  needed). 

Column  2  (Activity  Code)  -  Use  the  codes  at  the  top  of  the  page  to  indicate  the  type  of  activity.  If  more 
than  one  line  is  needed  to  describe  the  activity,  there  is  no  need  to  write  the  activity  code  for  each  line. 

Column  3  (Time)  Note  the  time  that  begins  each  5  minute  observation  period  (see  details  of  time¬ 
sampling  observation  procedure  below). 

Column  4  (Description/Topics  covered)  -  This  space  should  be  used  for  detailed  notes  about  the  student 
behaviors.  Detail  is  important;  it’s  better  to  write  down  too  much  than  too  little.  Use  this  space  to: 

•  Briefly  describe  (two  or  three  words)  the  nature  of  the  instruction.  It  is  important  to  link  the  student 
behaviors  with  specific  instruction  -  however,  the  person  observing  the  instruction  at  Camp 
Dodge  will  record  instructor  activities  in  detail. 

•  Describe  the  student  behaviors  in  detail  -  especially  when  the  Other  student  Behaviors  (OB) 
category  is  noted. 

•  Note  when  only  single  or  multiple  students  are  performing  such  behaviors. 

•  Provide  examples  of  the  student/student  or  instructor/student  coaching  or  tutoring  that  takes 
place. 

•  Explain  any  linkage  between  the  current  activity/task  and  previous  activities/tasks. 

•  Note  if  some  steps/points  had  to  be  presented  multiple  times. 

•  Capture  clarifying  questions  asked  by  students. 

•  Record  other  pertinent  information  that  will  assist  in  understanding  what  transpired  during  the 
activity. 
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Time-Sampling  Procedure:  To  simplify  the  quantification  of  student  behaviors,  we  will  use  a  time¬ 
sampling  procedure  in  which  we  will  note  whether  or  not  a  behavior  occurred  within  5-minute  intervals. 
During  any  given  5-minute  interval,  record  a  description  of  the  behavior  being  performed  and  any  relevant 
student  and  instructor  behaviors  in  column  4.  In  column  5  (this  can  be  done  in  real-time  or  even  at  a  later 
time)  note  which  behaviors  were  exemplified  in  each  row  of  column  4  (see  the  example  Student 
Observation  Form  below). 

Using  this  sampling  procedure,  you  can  aggregate  instances  of  each  student  behavior.  For  example  if  an 
instructor  spends  10  minutes  in  a  guided  demo  explaining  how  to  send  an  overlay,  you  would  describe 
the  student  behaviors  that  were  occurring  during  this  time  and  rather  than  tally  the  number  of  students 
who  were  performing  this  behavior  simply  note  “AQ”  in  the  5th  column  for  each  5  min  interval  that  the 
instructor  was  doing  the  guided  demo.  If  multiple  students  were  asking  questions,  then  you  could  note 
this  in  the  description  column,  column  4. 

You  may  note  more  than  one  student  behavior  occurring  during  a  five  minute  interval,  especially  if  the 
instruction  changes.  For  example,  an  instructor  may  end  one  guided  demo  and  then  spend  a  couple  of 
minutes  answering  questions  and  then  begin  another  guided  demo  within  one  5  min  observation  interval. 
Go  ahead  and  note  all  of  the  behaviors  that  occur  during  this  timeframe.  It’s  okay  to  record  behaviors  at 
a  higher  level  of  precision  than  we  will  actually  report  them. 

If  an  activity  spans  across  two  time  intervals,  for  example  if  a  guided  demo  lasts  more  than  5  minutes, 
please  indicate  that  this  is  the  same  demo  as  in  the  previous  time  interval.  Flowever,  you  may  use  “ditto”  if 
the  student  behaviors  are  the  same  (see  the  example  Student  Observation  Form  below) 

Part  III,  Summary  Questions.  The  Summary  questions  should  be  answered  after  each  observation 
period. 
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Part  II  Activity  Chronology 


Student  Behaviors 

OB  -  Other  student  Behaviors  are  observed 

SM  -  Instructor  takes  over  Student’s  Mouse 

AR  -  Asks  instructor  to  Repeat  step 

NP  -  Multiple  students  do  Not  Pay  attention  to 
the  lecture/demonstration 

UO  -  Uses  Other  functions  than  those  for  current  task 

AQ  -  Asks  instructor  a  Question 

PA  -  Pays  attention  to  lecture/PowerPoint 
presentation/Demonstration 

WF  -  Multiple  students  Waits  for  others  to  Finish 
task/PE 

GS  -  Works  in  Groups  to  Solve  individual  tasks 

PC  -  Peer  Coaching  (Seeks/Gives  help) 

WA  -  Works  Ahead  of  instruction  but  on  same  task 

IS  -  Works  Independently  to  Solve  problems 

RQ  -  Responds  to  instructor  Questions 

LB  -  Lags  Behind  instruction 

SC  -  Side  Conversations  during  instruction 

Num  =  Number  the  classroom  activities  (Activity 
Codes)  sequentially 

Activity  Code  =  Use  the  codes  at  the  top  of  the  page 
to  indicate  the  instructional  activity 

Time  =  Record  student  behaviors  for  every  5- 
minute  block  of  instruction 

Descriptions  =  Capture  examples  of  behavior; 
if  OB  is  used  please  describe  behavior  in  detail 

Student  Behaviors  =  Use  codes  of  student  behaviors; 
If  behavior  is  observed  once  in  a  5-minute  period  then 
record  the  behavior;  If  multiple  students  are  observed 
performing  the  behavior  then  note  that  in  the 
Description  column. 

Num 

Activity 

Code 

Time 

Description  of  topics  covered  and  instructor/student  behaviors 

Student 

Behaviors 

1 

LEC 

0800 

Lecture  on  maps.  Students  listen  to  lecture,  pay  attention  to  PowerPoint  slides. 

PA 

0805 

Con't  lecture  maps,  ditto 

PA 

0810 

Con't  lecture  maps,  ditto 

PA 

0815 

Con't  lecture  maps,  ditto 

PA 

2 

DEM 

0820 

Steps  to  build  3D  maps.  Two  students  asked  clarifying  questions. 

AQ 

0825 

Con't  Demo  3D  maps.  Student  works  ahead  of  instructor  and  builds  his  own  3D  map. 

WA 

3 

G/D 

0830 

G/D  3D  maps.  Several  students  cannot  keep  up  with  the  instructor. 

LB 

0835 

Con't  G/D  3D  maps.  Instructor  takes  over  mouse  of  one  student  to  show  him  the  steps  of 
building  a  3D  map. 

SM 

ACTIVITY  CODE 

VID  -  video 

DEM  -  demonstration 

G/D  -  guided  demo 

PE  -  practical  exercise 

REV  -  review 

BRK  -  break 

TEST  -quiz,  exam,  etc 

LEC  -  lecture 

SHEET  NUMBER: _ 1 

Date:  9  Feb  2006 
#  of  Students:  20 _ 


SHEET  NUMBER:  2 

Date: _ 9  Feb  2006 


Num 

Activity 

Code 

Time 

Description  of  topics  covered  and  instructor/student  behaviors 

Student 

Behaviors 

S/D 

0840 

Con't  S/D  3D  maps.  Several  students  finish  ahead  of  the  instructor. 

WF 

4 

S/D 

0845 

S/D  to  locate  and  select  a  map.  Several  students  ask  the  instructor  to  repeat  several  steps. 

Ak 

0850 

Con't  S/D  on  locate  and  select  a  map.  Student  creates  text  messages. 

U  O 

0855 

Con't  S/D  on  locate  and  select  a  map.  Students  ask  and  receive  help  from  fellow  students  on 
selecting  a  map. 

PC 

5 

BRK 

0900 

Break 

0905 

Break 

6 

S/D 

0910 

S/D  on  selecting  “Battle  Mode"  and  book  marking.  Several  students  ask  questions  about 
book  marking. 

AQ 

0915 

Con't  S/D  on  book  marking.  Several  students  talk  in  pairs. 

SC 

7 

PE 

0920 

PE  on  creating  and  sending  messages.  Students  work  individually  and  in  groups  to  complete 
the  individual  tasks  specified  by  the  PE. 

IS,  GS 

0925 

Con't  PE  on  messages.  Several  students  ask  the  instructor  questions.  Several  students  ask 

AQ,  PC 

0930 

Con't  PE  on  messages.  As  students  complete  the  PE,  they  use  other  FBCB2  functions. 

WF,  UO 

0935 

Con't  PE  on  messages.  One  student  requests  additional  help  from  the  instructor. 

LB 

APPENDIX  F 


Student  Survey  Results 


Table  FI 

Results  of  Student  Distributed  Learning  Survey 


Items 

Frequency  (Percent) 

Did  Not  Learn  / 
Partially  Learned 

Learned  All  Steps  /  Became 
Proficient 

Class  1  (n  =  10);  Class  2  (n  =  6) 

FBCB2-1 

FBCB2-2 

FBCB2-1 

FBCB2-2 

Extent  learned  the  system 

6  (60%) 

0  (0%) 

4  (40%) 

5  (83.3%) 

Hindered  Somewhat  / 
Hindered  Greatly 

Did  not  Hinder  nor 
Facilitate 

Facilitated  Somewhat  / 
Facilitated  Greatly 

FBCB2-1 

FBCB2-2 

FBCB2-1 

FBCB2-2 

FBCB2-1 

FBCB2-2 

Extent  distributed  format  affected  learning 

3  (30%) 

1  (16.7%) 

6  (60%) 

2  (33.3%) 

1  (10%) 

3  (50%) 

Somewhat  Less  Effective  / 
Much  Less  Effective 

About  the  Same 

Somewhat  More  Effective  / 
Much  More  Effective 

Comparison  to  other  distributed  learning  courses 

0  (0%) 

0  (0%) 

2  (20%) 

2  (33.3%) 

1  (10%) 

0  (0%) 

Comparison  to  resident  ABCS  courses 

1  (10%) 

1  (16.7%) 

3  (30%) 

3  (50%) 

0  (0%) 

1  (16.7%) 

Somewhat  Slow  / 
Extremely  Slow 

Just  Right 

A  Little  Fast-Paced  / 
Extremely  Fast-Paced 

Pace  of  instruction 

0  (0%) 

2  (33.3%) 

5  (50%) 

3  (50%) 

3  (30%) 

0  (0%) 

Never 

Only  duri 
To| 

ng  Certain 
aics 

Throughout  Entire  Course 

Worked  ahead  of  the  instruction 

4  (40%) 

4  (66.7%) 

4  (40%) 

1  (16.7%) 

1  (10%) 

1  (16.7%) 

Sought  assistance  from  fellow  students 

0  (0%) 

1  (16.7%) 

7  (70%) 

3  (50%) 

3  (30%) 

2  (33.3%) 

Assisted  other  students 

1  (10%) 

0  (0%) 

7  (70%) 

3  (50%) 

2  (20%) 

3  (50%) 

F-2 


None 

Some 

All 

FBCB2-1 

FBCB2-2 

FBCB2-1 

FBCB2-2 

FBCB2-1 

FBCB2-2 

Extent  instructor  answered  all  questions  and 
provided  clarification  of  steps  and  procedures 

0  (0%) 

0  (0%) 

1  (10%) 

1  (16.7%) 

9  (90%) 

5  (83.3%) 

Not 

Somewhat 

Very  /  Extremely 

Extent  engaged  and  motivated  to  learn  throughout 
the  course 

0  (0%) 

0  (0%) 

2  (20%) 

2  (33.3%) 

8  (80%) 

4  (66.7%) 

Student  comments  for  the  scaled  responses  above  included  the  following: 

•  Instruction  was  too  fast  only  when  computers  were  slow. 

•  Transceiver  Mgt  too  fast/unnecessary  -  should  be  a  field  rep  or  S6. 

•  When  I  needed  help,  instructor  took  over  and  just  started  hitting  buttons. 

•  Recommended  using  one  map  or  area  of  the  world. 

•  Without  an  instructor  on-site,  we  were  able  to  help  each  other  work  through  issues  which  was  good  for  the  team  and  increased 
learning. 

•  Tried  to  relate  what  was  being  done  to  something  that  they  know,  i.e.  setting  up  an  address  book,  setting  up  their  x-box,  etc. 

•  Great  instructor!  He  asked  questions/repeated  steps/functions. 

•  Mobilization  pending  is  #1.  Classroom  was  conducive  to  learning  -  individual  monitor/stations  and  instructor  motivation  and 
interest. 

•  It  was  good  training.  I  wasn't  familiar  with  computers. 

•  It  was  just  right.  Keep  it  the  same. 


Table  F2 

Ease  of  Learning  Different  FBCB2  Specific  Functions  or  Procedures 


FBCB2 

Functions  and 
Procedures 
Frequency  (Percent) 

Very 

Difficult 

Somewhat 

Difficult 

Neither  Difficult 

nor 

Easy 

Somewhat 

Easy 

Very  Easy 

FBCB2-1 

FBCB2-2 

FBCB2-1 

FBCB2-2 

FBCB2-1 

FBCB2-2 

FBCB2-1 

FBCB2-2 

FBCB2-1 

FBCB2-2 

Prep  for  Combat 

Configure  Role;  Clear 
Logs  and  Queues,  etc 

0  (0%) 

0  (0%) 

1  (10%) 

1  (16.7%) 

2  (20%) 

3  (50%) 

5  (50%) 

1  (16.7%) 

2  (20%) 

1  (16.7%) 

Messages 

Prepare/send  Cbt 

Msg;  Spot  Report; 
Addressing,  etc 

0  (0%) 

0  (0%) 

2  (20%) 

0  (0%) 

1  (10%) 

4  (66.7%) 

6  (60%) 

2  (33.3%) 

1  (10%) 

0  (0%) 

Navigation  Tools 

CLOS;  LOS;  Periodic 
Reminders,  etc 

0  (0%) 

0  (0%) 

3  (30%) 

0  (0%) 

4  (40%) 

4  (66.7%) 

3  (30%) 

1  (16.7%) 

0  (0%) 

1  (16.7%) 

Orders 

Prepare  and  send 
Warno  /  Opord; 

Attach  Overlay,  etc 

0  (0%) 

0  (0%) 

4  (40%) 

1  (16.7%) 

3  (30%) 

3  (50%) 

3  (30%) 

2  (33.3%) 

0  (0%) 

0  (0%) 

Overlay  Messages 

Create  Objects; 
Save/Send  Overlay, 
etc 

0  (0%) 

0  (0%) 

6  (60%) 

1  (16.7%) 

2  (20%) 

3  (50%) 

2  (20%) 

1  (16.7%) 

0  (0%) 

1  (16.7%) 
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Table  F3 

Effectiveness  of  Different  Instructional  Activities 


Instructional 

Activities 

Very 

Ineffective 

Somewhat 

Ineffective 

Neither  Ineffective  or 
Effective 

Somewhat 

Effective 

Very  Effective 

FBCB2-1 

FBCB2-2 

FBCB2-1 

FBCB2-2 

FBCB2-1 

FBCB2-2 

FBCB2-1 

FBCB2-2 

FBCB2-1 

FBCB2-2 

Demonstration  of 
steps  and 
procedures 
without  the 
students  using 
the  system  to 
follow  along 

4  (40%) 

0  (0%) 

1  (10%) 

1  (16.7%) 

0  (0%) 

1  (16.7%) 

5  (50%) 

3  (50%) 

0  (0%) 

1  (16.7%) 

Guided 

demonstration  of 
steps  and 
procedures  with 
the  students 
using  the  system 
to  follow  along 

0  (0%) 

0  (0%) 

0  (0%) 

1  (16.7%) 

0  (0%) 

0  (0%) 

7  (70%) 

3  (50%) 

3  (30%) 

2  (33.3%) 

Lecture  / 

PowerPoint  slides 

0  (0%) 

1  (16.7%) 

2  (20%) 

0  (0%) 

3  (30%) 

1  (16.7%) 

4  (40%) 

2  (33.3%) 

1  (10%) 

2  (33.3%) 

Practical  Exercise 

0  (0%) 

0  (0%) 

0  (0%) 

0  (0%) 

2  (20%) 

1  (16.7%) 

3  (30%) 

1  (16.7%) 

5  (50%) 

4  (66.7%) 

Video 

0  (0%) 

0  (0%) 

1  (10%) 

0  (0%) 

4  (40%) 

1  (16.7%) 

4  (40%) 

0  (0%) 

0  (0%) 

2  (33.3%) 

Review  of 
material  that  was 
covered 

0  (0%) 

0  (0%) 

0  (0%) 

0  (0%) 

0  (0%) 

1  (16.7%) 

7  (70%) 

3  (50%) 

3  (30%) 

2  (33.3%) 

Test  -  quiz,  exam 

0  (0%) 

0  (0%) 

0  (0%) 

0  (0%) 

2  (20%) 

0  (0%) 

5  (50%) 

2  (33.3%) 

3  (30%) 

3  (50%) 
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ACRONYMS  AND  ABBREVIATIONS 


ABCS 

Army  Battle  Command  System 

AI 

Assistant  Instructor 

ALM 

Army  Learning  Model 

AQ 

Asks  Instructor  a  Question 

ARFORGEN 

Army  Force  Generation  Model 

AR 

Asks  Instructor  to  Repeat  Step 

ARI 

U.  S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences 

BCTC 

Battle  Command  Training  Center 

BCT&DS 

Battle  Command  Training  and  Distributed  System 

BRK 

Break 

CD 

compact  disc 

DEM 

Demonstration 

dL 

Distributed  Learning 

EA 

Encourages  Active  Learning 

EP 

Emphasizes  Practice 

FBCB2 

Force  XXI  Battle  Command  Brigade  and  Below 

F2F 

Face-to-Face 

G/D 

Guided  Demonstration 

GS 

Works  in  Groups  to  Solve  Individual  Tasks 

IS 

Works  Independently  to  Solve  Problems 

LB 

Lags  Behind  instruction 

LC 

Learning  Check 

LEC 

Lecture 

MA 

Uses  Memory  Aids 

MC3-RC 

Maneuver  Captains  Career  Course  for  Army  National  Guard  or  Army 
Reserve  Soldiers 

MEDEVAC 

Medical  Evacuation 

MTTs 

Mobile  Training  Teams 

NP 

Does  Not  Pay  Attention  to  the  Lecture/Demonstration 

OB 

Other  Student  Behaviors  Are  Observed 

PA 

Pays  Attention  to  Lecture/PowerPoint  Presentation/Demonstration 

PC 

Peer  Coaching 

PE 

Practical  Exercise 

POI 

Program  of  Instruction 

PP 

Path  and  Purpose 

RQ 

Responds  to  Instructor  Questions 

REV 

review 

RL 

Respond  to  Learners 

RG 

Relate  to  General  Knowledge 

RM 

Relate  to  Military  Operations 

RP 

Relate  to  Previous  Content 

SC 

Side  Conversations  During  Instruction 
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SM 

Instructor  takes  over  Student’s  Mouse 

SP 

Screen  Prompts 

SMEs 

Subject  Matter  Experts 

TTP 

Tactics,  techniques,  and  procedures 

TRADOC 

U.S.  Army  Training  and  Doctrine  Command 

UO 

Uses  Other  Functions  Than  Those  for  Current  Task 

VID 

Video 

VTC 

Video  Teleconferencing 

VTT 

Video  Tele-Training 

WF 

Waits  for  Others  to  Finish  Task/PE 

WA 

Works  Ahead  of  Instruction  but  on  Same  Task 
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